object UnwrapCastInBinaryComparison extends Rule[LogicalPlan]
Unwrap casts in binary comparison or In/InSet operations with patterns like following:
- BinaryComparison(Cast(fromExp, toType), Literal(value, toType))
- BinaryComparison(Literal(value, toType), Cast(fromExp, toType))
- In(Cast(fromExp, toType), Seq(Literal(v1, toType), Literal(v2, toType), ...)
- InSet(Cast(fromExp, toType), Set(v1, v2, ...))
This rule optimizes expressions with the above pattern by either replacing the cast with simpler constructs, or moving the cast from the expression side to the literal side, which enables them to be optimized away later and pushed down to data sources.
Currently this only handles cases where:
1). fromType (of fromExp) and toType are of numeric types (i.e., short, int, float,
decimal, etc) or boolean type
2). fromType can be safely coerced to toType without precision loss (e.g., short to int,
int to long, but not long to int, nor int to boolean)
If the above conditions are satisfied, the rule checks to see if the literal value is within
range (min, max), where min and max are the minimum and maximum value of fromType,
respectively. If this is true then it means we may safely cast value to fromType and thus
able to move the cast to the literal side. That is:
cast(fromExp, toType) op value ==> fromExp op cast(value, fromType)
Note there are some exceptions to the above: if casting from value to fromType causes
rounding up or down, the above conversion will no longer be valid. Instead, the rule does the
following:
if casting value to fromType causes rounding up:
cast(fromExp, toType) > value==>fromExp >= cast(value, fromType)cast(fromExp, toType) >= value==>fromExp >= cast(value, fromType)cast(fromExp, toType) === value==> if(isnull(fromExp), null, false)cast(fromExp, toType) <=> value==> false (iffromExpis deterministic)cast(fromExp, toType) <= value==>fromExp < cast(value, fromType)cast(fromExp, toType) < value==>fromExp < cast(value, fromType)
Similarly for the case when casting value to fromType causes rounding down.
If the value is not within range (min, max), the rule breaks the scenario into different
cases and try to replace each with simpler constructs.
if value > max, the cases are of following:
cast(fromExp, toType) > value==> if(isnull(fromExp), null, false)cast(fromExp, toType) >= value==> if(isnull(fromExp), null, false)cast(fromExp, toType) === value==> if(isnull(fromExp), null, false)cast(fromExp, toType) <=> value==> false (iffromExpis deterministic)cast(fromExp, toType) <= value==> if(isnull(fromExp), null, true)cast(fromExp, toType) < value==> if(isnull(fromExp), null, true)
if value == max, the cases are of following:
cast(fromExp, toType) > value==> if(isnull(fromExp), null, false)cast(fromExp, toType) >= value==> fromExp == maxcast(fromExp, toType) === value==> fromExp == maxcast(fromExp, toType) <=> value==> fromExp <=> maxcast(fromExp, toType) <= value==> if(isnull(fromExp), null, true)cast(fromExp, toType) < value==> fromExp =!= max
Similarly for the cases when value == min and value < min.
Further, the above if(isnull(fromExp), null, false) is represented using conjunction
and(isnull(fromExp), null), to enable further optimization and filter pushdown to data sources.
Similarly, if(isnull(fromExp), null, true) is represented with or(isnotnull(fromExp), null).
For In/InSet operation, first the rule transform the expression to Equals:
Seq(
EqualTo(Cast(fromExp, toType), Literal(v1, toType)),
EqualTo(Cast(fromExp, toType), Literal(v2, toType)),
...
)
and using the same rule with BinaryComparison show as before to optimize each EqualTo.
- Alphabetic
- By Inheritance
- UnwrapCastInBinaryComparison
- Rule
- Logging
- SQLConfHelper
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def apply(plan: LogicalPlan): LogicalPlan
- Definition Classes
- UnwrapCastInBinaryComparison → Rule
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def conf: SQLConf
The active config object within the current scope.
The active config object within the current scope. See SQLConf.get for more information.
- Definition Classes
- SQLConfHelper
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- lazy val ruleId: RuleId
- Attributes
- protected
- Definition Classes
- Rule
- val ruleName: String
Name for this rule, automatically inferred based on class name.
Name for this rule, automatically inferred based on class name.
- Definition Classes
- Rule
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()