From 5967bb5388f7f3a17cf2cb3993993a6425d7318f Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Thu, 4 Jan 2024 09:38:49 -0500 Subject: [PATCH 1/2] Minor: Improve library docs to mention TreeNode, ExprSimplifier, PruningPredicate and cp_solver --- datafusion/core/src/lib.rs | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/datafusion/core/src/lib.rs b/datafusion/core/src/lib.rs index b3ebbc6e3637e..179f5550a77d2 100644 --- a/datafusion/core/src/lib.rs +++ b/datafusion/core/src/lib.rs @@ -285,37 +285,38 @@ //! //! ### Logical Plans //! Logical planning yields [`LogicalPlan`] nodes and [`Expr`] -//! expressions which are [`Schema`] aware and represent statements +//! representing expressions which are [`Schema`] aware and represent statements //! independent of how they are physically executed. //! A [`LogicalPlan`] is a Directed Acyclic Graph (DAG) of other //! [`LogicalPlan`]s, each potentially containing embedded [`Expr`]s. //! -//! Examples of working with and executing `Expr`s can be found in the +//! [`Expr`]s can be rewritten using the [`TreeNode`] API and simplified using +//! [`ExprSimplifier`]. Examples of working with and executing `Expr`s can be found in the //! [`expr_api`.rs] example //! +//! [`TreeNode`]: datafusion_common::tree_node::TreeNode +//! [`ExprSimplifier`]: datafusion::optimizer::simplify_expressions::ExprSimplifier //! [`expr_api`.rs]: https://github.com/apache/arrow-datafusion/blob/main/datafusion-examples/examples/expr_api.rs //! //! ### Physical Plans //! //! An [`ExecutionPlan`] (sometimes referred to as a "physical plan") //! is a plan that can be executed against data. It a DAG of other -//! [`ExecutionPlan`]s each potentially containing expressions of the -//! following types: +//! [`ExecutionPlan`]s each potentially containing expressions that implement the +//! [`PhysicalExpr`] trait. //! -//! 1. [`PhysicalExpr`]: Scalar functions -//! -//! 2. [`AggregateExpr`]: Aggregate functions -//! -//! 2. [`WindowExpr`]: Window functions -//! -//! Compared to a [`LogicalPlan`], an [`ExecutionPlan`] has concrete +//! Compared to a [`LogicalPlan`], an [`ExecutionPlan`] has additional concrete //! information about how to perform calculations (e.g. hash vs merge //! join), and how data flows during execution (e.g. partitioning and //! sortedness). //! +//! [cp_solver] performs range propagation analysis on [`PhysicalExpr`]s and +//! [`PruningPredicate`] can prove certain boolean [`PhysicalExpr`]s used for +//! filtering can never be `true` using additional statistical information. +//! +//! [cp_solver]: crate::physical_expr::intervals::cp_solver +//! [`PruningPredicate`]: crate::physical_optimizer::pruning::PruningPredicate //! [`PhysicalExpr`]: crate::physical_plan::PhysicalExpr -//! [`AggregateExpr`]: crate::physical_plan::AggregateExpr -//! [`WindowExpr`]: crate::physical_plan::WindowExpr //! //! ## Execution //! From 20f9ca6e492792a72baa9d7b01062ac5345d4b4d Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Thu, 4 Jan 2024 11:39:04 -0500 Subject: [PATCH 2/2] fix link --- datafusion/core/src/lib.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/datafusion/core/src/lib.rs b/datafusion/core/src/lib.rs index 179f5550a77d2..8fc724a22443f 100644 --- a/datafusion/core/src/lib.rs +++ b/datafusion/core/src/lib.rs @@ -295,7 +295,7 @@ //! [`expr_api`.rs] example //! //! [`TreeNode`]: datafusion_common::tree_node::TreeNode -//! [`ExprSimplifier`]: datafusion::optimizer::simplify_expressions::ExprSimplifier +//! [`ExprSimplifier`]: crate::optimizer::simplify_expressions::ExprSimplifier //! [`expr_api`.rs]: https://github.com/apache/arrow-datafusion/blob/main/datafusion-examples/examples/expr_api.rs //! //! ### Physical Plans