Spark Cassandra Connector

Lightning-fast cluster computing with Spark and Cassandra

This library lets you expose Cassandra tables as Spark RDDs, write Spark RDDs to Cassandra tables, and execute arbitrary CQL queries in your Spark applications.

Features

  • Compatible with Apache Cassandra version 2.0 or higher and DataStax Enterprise 4.5
  • Compatible with Apache Spark 1.0 and 1.1
  • Exposes Cassandra tables as Spark RDDs
  • Maps table rows to CassandraRow objects or tuples
  • Offers customizable object mapper for mapping rows to objects of user-defined classes
  • Saves RDDs back to Cassandra by implicit saveToCassandra call
  • Converts data types between Cassandra and Scala
  • Supports all Cassandra data types including collections
  • Filters rows on the server side via the CQL WHERE clause
  • Allows for execution of arbitrary CQL statements
  • Plays nice with Cassandra Virtual Nodes

Download

This project has been published to the Maven Central Repository. For SBT to download the connector binaries, sources and javadoc, put this in your project SBT config:

libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "1.1.0"

If you want to access the functionality of Connector from Java, you may want to add also a Java API module:

libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector-java" % "1.1.0"

Building

Building The Assembly Jar

In the root directory run

sbt assembly

A fat jar will be generated to both of these directories:

  • spark-cassandra-connector/target/scala-2.10/
  • spark-cassandra-connector-java/target/scala-2.10/

Select the former for Scala apps, the later for Java.

Building General Artifacts

In the root directory run:

sbt package
sbt doc

The library package jars will be placed in:

  • spark-cassandra-connector/target/scala-2.10/
  • spark-cassandra-connector-java/target/scala-2.10/

The documentation will be generated to:

  • spark-cassandra-connector/target/scala-2.10/api/
  • spark-cassandra-connector-java/target/scala-2.10/api/
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s