A custom task metric. This is a logical representation of a metric reported by data sources at the executor side. During query execution, Spark will collect the task metrics per partition by PartitionReader and update internal metrics based on collected metric values. For streaming query, Spark will collect and combine metrics for a final result per micro batch.

The metrics will be gathered during query execution back to the driver and then combined. How the task metrics are combined is defined by corresponding CustomMetric with same metric name. The final result will be shown up in the data source scan operator in Spark UI.

There are a few special metric names: "bytesWritten" and "recordsWritten". If the data source defines custom metrics with the same names, the metric values will also be updated to corresponding task metrics.