Spark
- Spark
- Overview
- Important requirement
- Run applications located on S3
- Add S3 libraries and credentials
- Access the Driver Web UI
- Persist SparkApplication state
- Use Spark from Jupyter Notebook
- Access S3 from a Jupyter Notebook
- Example SparkApplication using code from S3
- Example notebook code for reading and writing S3 data
- What to do next
- Change the namespace for a SparkApplication
- Spark test scenarios
- Overview
- Important notes
- Scenario 1. Successful execution of a Spark job from example.jar on S3
- Scenario 2. Resource constraints prevent Spark job execution
- Scenario 3. Spark works correctly when launched from a notebook
- Scenario 4. Missing example.jar on S3 prevents Spark job execution
- Scenario 5. Missing or incorrect S3 secret prevents job execution
- Scenario 6. Spark job error caused by a non-existent main class
- What to do next
- Delete a SparkApplication