Class

com.databricks.spark.csv

CsvRelation

Related Doc: package csv

Permalink

case class CsvRelation(baseRDD: () ⇒ RDD[String], location: Option[String], useHeader: Boolean, delimiter: Char, quote: Character, escape: Character, comment: Character, parseMode: String, parserLib: String, ignoreLeadingWhiteSpace: Boolean, ignoreTrailingWhiteSpace: Boolean, treatEmptyValuesAsNulls: Boolean, userSchema: StructType = null, inferCsvSchema: Boolean, codec: String = null, nullValue: String = "", dateFormat: String = null, maxCharsPerCol: Int = 100000)(sqlContext: SQLContext) extends BaseRelation with TableScan with PrunedScan with InsertableRelation with Product with Serializable

Linear Supertypes
Serializable, Serializable, Product, Equals, InsertableRelation, PrunedScan, TableScan, BaseRelation, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. CsvRelation
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. InsertableRelation
  7. PrunedScan
  8. TableScan
  9. BaseRelation
  10. AnyRef
  11. Any
  1. Hide All
  2. Show all
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CsvRelation(baseRDD: () ⇒ RDD[String], location: Option[String], useHeader: Boolean, delimiter: Char, quote: Character, escape: Character, comment: Character, parseMode: String, parserLib: String, ignoreLeadingWhiteSpace: Boolean, ignoreTrailingWhiteSpace: Boolean, treatEmptyValuesAsNulls: Boolean, userSchema: StructType = null, inferCsvSchema: Boolean, codec: String = null, nullValue: String = "", dateFormat: String = null, maxCharsPerCol: Int = 100000)(sqlContext: SQLContext)

    Permalink
    Attributes
    protected[com.databricks.spark]

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  5. val baseRDD: () ⇒ RDD[String]

    Permalink

  6. def buildScan(requiredColumns: Array[String]): RDD[Row]

    Permalink

    This supports to eliminate unneeded columns before producing an RDD containing all of its tuples as Row objects.

    This supports to eliminate unneeded columns before producing an RDD containing all of its tuples as Row objects. This reads all the tokens of each line and then drop unneeded tokens without casting and type-checking by mapping both the indices produced by requiredColumns and the ones of tokens.

    Definition Classes
    CsvRelation → PrunedScan

  7. def buildScan(): RDD[Row]

    Permalink
    Definition Classes
    CsvRelation → TableScan

  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  9. val codec: String

    Permalink

  10. val comment: Character

    Permalink

  11. val dateFormat: String

    Permalink

  12. val delimiter: Char

    Permalink

  13. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  14. val escape: Character

    Permalink

  15. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  16. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  17. val ignoreLeadingWhiteSpace: Boolean

    Permalink

  18. val ignoreTrailingWhiteSpace: Boolean

    Permalink

  19. val inferCsvSchema: Boolean

    Permalink

  20. def insert(data: DataFrame, overwrite: Boolean): Unit

    Permalink
    Definition Classes
    CsvRelation → InsertableRelation

  21. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  22. val location: Option[String]

    Permalink

  23. val maxCharsPerCol: Int

    Permalink

  24. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  25. def needConversion: Boolean

    Permalink
    Definition Classes
    BaseRelation

  26. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  27. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  28. val nullValue: String

    Permalink

  29. val parseMode: String

    Permalink

  30. val parserLib: String

    Permalink

  31. val quote: Character

    Permalink

  32. val schema: StructType

    Permalink
    Definition Classes
    CsvRelation → BaseRelation

  33. def sizeInBytes: Long

    Permalink
    Definition Classes
    BaseRelation

  34. val sqlContext: SQLContext

    Permalink
    Definition Classes
    CsvRelation → BaseRelation

  35. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  36. val treatEmptyValuesAsNulls: Boolean

    Permalink

  37. def unhandledFilters(filters: Array[Filter]): Array[Filter]

    Permalink
    Definition Classes
    BaseRelation

  38. val useHeader: Boolean

    Permalink

  39. val userSchema: StructType

    Permalink

  40. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  41. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  42. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from InsertableRelation

Inherited from PrunedScan

Inherited from TableScan

Inherited from BaseRelation

Inherited from AnyRef

Inherited from Any

Ungrouped