diff --git a/docs/structured-streaming-programming-guide.md b/docs/structured-streaming-programming-guide.md index da5c2344a1b18..e536f5d8ecbd5 100644 --- a/docs/structured-streaming-programming-guide.md +++ b/docs/structured-streaming-programming-guide.md @@ -449,6 +449,10 @@ Here are the details of all the sources in Spark. File source path: path to the input directory, and common to all file formats. +
+ maxFilesPerTrigger: maximum number of new files to be considered in every trigger (default: no max) +
+ latestFirst: whether to processs the latest new files first, useful when there is a large backlog of files(default: false)

For file-format-specific options, see the related methods in DataStreamReader (Scala/Java/Python). @@ -1076,9 +1080,6 @@ Here are the details of all the sinks in Spark. Append path: path to the output directory, must be specified. - maxFilesPerTrigger: maximum number of new files to be considered in every trigger (default: no max) -
- latestFirst: whether to processs the latest new files first, useful when there is a large backlog of files(default: false)

For file-format-specific options, see the related methods in DataFrameWriter (Scala/Java/Python).