flexmeasures.cli.jobs
CLI commands for controlling jobs
Functions
- flexmeasures.cli.jobs.get_exception_handler(queue_name: str) Callable
Look up the exception handler to use for jobs on a given queue.
Queues without a handler of their own fall back to the generic handler.
- Parameters:
queue_name – the name of a single queue, e.g. ‘forecasting’.
- Returns:
an RQ exception handler.
- flexmeasures.cli.jobs.handle_exception_by_queue(job: Job, exc_type: Type[Exception], exc_value: Exception, traceback: TracebackType) bool | None
Handle a failed job with the exception handler of the queue the job came from.
A worker that works on several queues would otherwise use one handler for the jobs of all of them. RQ reads the return value to decide whether to call the next handler, so we pass it on.
- flexmeasures.cli.jobs.handle_worker_exception(job: Job, exc_type: Type[Exception], exc_value: Exception, traceback: TracebackType) None
Just a fallback, usually we would use the per-queue handler.
- flexmeasures.cli.jobs.parse_queue_list(queue_names_str: str) list[Queue]
Parse a | separated string of queue names against the app.queues dict.
The app.queues dict is expected to have queue names as keys, and rq.Queue objects as values.
- Parameters:
queue_names_str – a string with queue names separated by the | character
- Returns:
a list of Queue objects.
- flexmeasures.cli.jobs.release_database_connections_before_forking()
Keep forked job processes from sharing the worker’s database connections.
RQ’s Worker forks a process for every job, and a fork copies the parent’s connection pool, sockets included. A job would then talk to Postgres over the same session as the worker and as every job before it, so a job exiting mid-transaction would make the next job’s first query fail (#2806). Therefore, the worker gives up its own connections before it starts, and every forked process starts from an empty pool, leaving the parent’s sockets alone. See https://docs.sqlalchemy.org/en/20/core/pooling.html#using-connection-pools-with-multiprocessing-or-os-fork.