 |
||||||||
|
||||||||
Spark Driver Program - Python Automation and Machine Learning for ICs - - An Online Book: Python Automation and Machine Learning for ICs by Yougui Liao - |
||||||||
| Python Automation and Machine Learning for ICs http://www.globalsino.com/ICs/ | ||||||||
| Chapter/Index: Introduction | A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | Appendix | ||||||||
================================================================================= The Spark driver program is software located on the main node of a machine. It outlines operations on RDDs (Resilient Distributed Datasets), detailing both transformations and actions. Essentially, the Spark driver starts a SparkContext that connects to a specific Spark Master. It also sends RDD graphs to the Master, where the standalone cluster manager functions. The driver process in an Apache Spark application is typically run on the cluster's client machine or the master node. However, note that the driver process itself doesn't execute the tasks; it coordinates the execution by communicating with the cluster manager and the executors. ===========================================
|
||||||||
| ================================================================================= | ||||||||
|
|
||||||||
