Troubleshooting Data Flow
Use troubleshooting information to identify and address common issues that can occur while working with Data Flow.
Resources Exceed Quota Limit
Fix problems where the resources requested exceed to quota limit.
The job doesn't run.
You get a message: The resources you have requested exceed the tenant's Data Flow quota limit.
Cause:
You have exceeded your quota of resources.
Remedy:
- In the Oracle Cloud Infrastructure Console, navigate to Governance.
- Select Limits, Quotas and Usage.
- Click Request a service limit increase, and request more quota for Data Flow.
Spark UI Fails to Load or Returns a 500 Internal Server Error
For in-progress Runs, the Spark UI might fail to load or give a 500 internal server error.
Whilst a Run is in progress, you can't load the Spark UI. It might return a 500 internal server error.
If the Driver is too busy running jobs or there are too many jobs to load, it might take longer for the UI to load and it might time out, resulting in an error.
Use a bigger driver shape
Reduce the retained jobs and stages
spark.ui.retainedjobs and
spark.ui.retainedstages are set to 1000. Set the values to
a lower number to try to avoid a timeout.DF-6000: Data Flow Service is Experiencing Capacity Limitation
An application run times out when running with a DF-6000 error.
A run times out with the following state details: DF-6000: Data Flow service is experiencing capacity limitation for <preferred shape family> shape. Please retry in a few minutes, or use a different shape. Please submit a support ticket if you need a limit increase for any shape.
This indicates insufficient compute capacity of the selected shape.
DF-7001: Your Cloud Tenant has Reached its Data Flow Quota Limit
An application run fails with a DF-7001 error.
A run fails with the following state details: DF-7001: Your cloud tenant has reached its Data Flow quota limit for <preferred shape family> shape. Contact your Administrator to increase the limits then resubmit the run or use a different VM shape.
This indicates that your tenancy has exceeded the quota limit for the preferred shape.
DF-7003: The Data Flow Service has Reached its Quota Limit
An application run fails with a DF-7003 error.
A run fails with the following state details: DF-7003: The Data Flow service has reached its quota limit. Please wait for runs to finish or contact Oracle to increase the limits.
This error code indicates that Data Flow has reached its quota limit.
Data Flow might already be working toward increasing the quota limit, but we request you contact Oracle Support to get the current status of when the quota limit is available for use.