The only thing I want to prevent are timeouts and errors in our logs. These are caused by uploadAndMatchContextSync
because it uses the wait
method to check status of the (slow?) matching process on Smartling side.
That's why I would split the functionality up in 2 steps/queues but keep all existing logic, that way we give Smartling the time it needs to finish processing without failing on our side:
- trigger
uploadAndMatchContext
, retrieve theprocessUid
and create a new item in a separate queue -
- queue worker uses
DelayedRequeueException
instead of waiting in the PHP process itself - when process is finished, fetch all information using
uploadAndMatchContext
again and log messages - trigger (improved)
uploadContextMissingResources
- queue worker uses
The major difference it that the waiting part is not done in PHP but in a queue with the correct exceptions, that won't block the queue. Processing of other queue items will continue while waiting for slow ones.
Part of my comment here https://www.drupal.org/project/tmgmt_smartling/issues/3546398 is related to this topic as well:
I think uploading all of these assets one by one also makes the process incredibly slow. Why not use Guzzle http client instead of curl commands directly after which you could easily execute async upload commands for all of those assets: https://docs.guzzlephp.org/en/stable/quickstart.html#concurrent-requests