Workflow is trigger infintely #176

demobiel · 2024-04-26T19:42:31Z

demobiel
Apr 26, 2024

Hi,

I am a newbie to this package.

When I use the command php artisan queue:work everything works as expected, but when I run horizon my workflow keeps being triggered, without executing any of the activties.

The horizon.log shows the following:

...
  2024-04-26 18:54:36 App\Workflows\MyWorkflow .................................................................................................................................................................................. RUNNING
  2024-04-26 18:54:36 App\Workflows\MyWorkflow ............................................................................................................................................................................. 14.42ms DONE
  2024-04-26 18:54:36 App\Workflows\MyWorkflow .................................................................................................................................................................................. RUNNING
  2024-04-26 18:54:36 App\Workflows\MyWorkflow ............................................................................................................................................................................. 14.49ms DONE
  2024-04-26 18:54:36 App\Workflows\MyWorkflow .................................................................................................................................................................................. RUNNING
...

I guess something is wrong with my horizon setup (however it works fine with other jobs), so I have no idea where to look.

The only clue I have until now, is that in the horizon UI the job remains in "pending".

Any help or a direction would be helpful.

Kind regards

Andries

This is my workflow

<?php

namespace App\Workflows;

use App\Models\Order;
use Workflow\ActivityStub;
use Workflow\Workflow;

class MyWorkflow extends Workflow
{
    public function execute()
    {
        $result = yield ActivityStub::make(MyActivity::class);
        return $result;
    }

}

and this is the activity


<?php

namespace App\Workflows;

use App\Models\Order;
use App\Services\Order\OrderEmailService;
use Workflow\Activity;

class MyActivity extends Activity
{
    public function execute(): string
    {
        // Perform some work...
        return "test";
    }
}

Answered by rmcdaniel

Apr 26, 2024

Did you check the logs for error messages?

View full answer

rmcdaniel · 2024-04-26T20:07:50Z

rmcdaniel
Apr 26, 2024
Maintainer

Did you check the logs for error messages?

1 reply

demobiel Apr 27, 2024
Author

there was nothing in the workflows_exception table, but there were entries in the failed_jobs.

Removing them solved the issue. However I don't understand why, and why the behaviour between php artisan queue:work and horizon is different in this case, but I guess that has more to do with my understanding of horizon then with this package.

thanks for your reply and this package!

jeanlinux · 2025-03-03T11:12:13Z

jeanlinux
Mar 3, 2025

Guys, this issue is real, I am facing the same thing. when I run laravel queue:work, it starts processing my workflows, usually the first one is processed and completed but when the next queue is being processed, I receive a similar error ..

$   2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 5.38ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.77ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 5.60ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.35ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.44ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.78ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.32ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.88ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.54ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 5.27ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.68ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.57ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 4.32ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.59ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 4.33ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.84ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.61ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.98ms DONE
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow .......... RUNNING
  2025-03-03 11:00:11 App\Workflows\LoadParseEnrichWorkflow ...... 3.72ms DONE.

Note: there are no errors in the error logs and also checking failed jobs in the database, it is empty, but for workflows, I see the last workflow with status pending, I am on development server with redis for running queue. this error persists until i ran php artisan migrate:fresh, and then the next immediate workflow will run in the queue and then this error comes back when you put something new in the queue.

i have a regular workflow.

<?php

namespace App\Workflows;

use Workflow\Workflow;
use Workflow\ActivityStub;
use App\Activities\EnrichActivity;
use App\Activities\LoadActivity;
use App\Activities\ParseActivity;
use Exception;


class LoadParseEnrichWorkflow extends Workflow
{
    // definition to execute the workflow
    public function execute(string $scanId, array $fileNames, array $params, array $ownerInfo, string $auth)
    {
        try{
            return [
                yield ActivityStub::make(LoadActivity::class, $scanId),
                yield ActivityStub::make(ParseActivity::class, $scanId, $fileNames, $params, $ownerInfo, $auth),
                yield ActivityStub::make(EnrichActivity::class, $scanId),
            ];
        }catch(Exception $e){
            Log::error($e->getMessage());
        }
            
        
    }
}

<?php

namespace App\Activities;

use App\Models\Scan;
use Workflow\Activity;
use Workflow\WorkflowStub;
use Carbon\Carbon;

class LoadActivity extends Activity
{
    // public $tries = 1;

    // Define the activity to run
    public function execute(string $scanId)
    {
        $result = [];

        // get the workflow ID and update scan record
        $workflowId = $this->workflowId();

        // // Update scan record, ensure its   idempotent
        $scan = Scan::findOrFail($scanId);

        // // Update workflow id only if not set
        if (is_null($scan->workflow_id)) {
            $scan->workflow_id = $workflowId;
        }

        // // Update scan status only if it has not run first time
        if ($scan->status === 'QUEUED') {
            $scan->status = 'PROCESSING';
        }

        // // Update workflow started at only if not set
        if (is_null($scan->workflow_started_at)) {
            // $scan->workflow_started_at = WorkflowStub::now();
            $scan->workflow_started_at = Carbon::now();
        }

        // // Save Scan
        $scan->save();

        $result = [
            'success' => true,
            'recordUpdate' => true,
            'recordId' => $scanId,
            'workflowId' => $workflowId
        ];

        return $result;
    }
}

<?php

namespace App\Activities;

use Workflow\Activity;
use App\Models\Scan;
use Jeanlinux\Component\MobileMoneyStatement\Parser\PDFParser;
use App\Services\Scanners\Documents\PDF\MobileMoney\DocumentParser;
use App\Services\Scanners\Documents\PDF\MobileMoney\MTNGH\Processor;
use Illuminate\Support\Facades\Storage;
use Exception;
use PDOException;
use Log;
use App\Helpers\ChargesHelper;



class ParseActivity extends Activity
{
    // public $tries = 1;

    private bool $ready = false;

    #[QueryMethod]
    public function getReady(): bool
    {
        return $this->ready;
    }

    // Define the activity to run
    public function execute(string $scanId, array $fileNames, array $params, array $ownerInfo = [], string $auth = null, bool $useFilesFromS3 = false)
    {
        // get the workflow ID
        $workflowId = $this->workflowId();

        // Update scan record, ensure its   idempotent
        $scan = Scan::findOrFail($scanId);
        $result = null;


        // Only update status if its Processing
        if ($scan->status === "PROCESSING") {
            $scan->status = "SCANNING";
            $scan->save();
        }

        // Check that we have the files ready for processing
        if ($useFilesFromS3) {
            // We are going to process the files from s3
            // Download files from s3 into our tmp folder
        }

        $results = array(
            "error_count" => 0,
            "successful" => 0,
            "persisted" => 0,
            "existing" => 0,
            "errors" => []
        );
        $includeRecords = $params['include_records'];

        // We parse the document based on the type
        // statements location uploads/scans/$ref
        switch ($scan->scan_type):
            case "MOBILEMONEY":
                // Scan money money statement(s)
                foreach ($fileNames as $key => $fileName) {
                    // Process each mobile money statement
                    try {

                        $path = "uploads/scans/{$scan->ref}/" . $fileName;
                        if (Storage::disk('local')->exists($path)) {
                            $file = Storage::disk('local')->path($path);
                        } else {
                            throw new Exception("Error: File to parse does not exist.");
                        }
                    } catch (Exception $e) {
                        Log::error($e->getMessage());
                        $results["error_count"] = $results["error_count"] + 1;
                        $results["errors"][] = ["error" => "Could not load file for scanning.", "error_code" => "001"];
                    }
                    $pdfParser = new PDFParser();
                    // Log::debug("File is {$file}");

                    $mobileMoneyStatementParser = new DocumentParser($pdfParser, $file);

                    // Send heartbeat
                    // $this->heartbeat();

                    // switch ($mobileMoneyStatementParser->getStatementProvider()):
                    //     case "MTNGH":

                    try {

                        // Process MTN GH statement
                        $statement = $mobileMoneyStatementParser->parseDocument();
                        // if ($statement === -1){
                        //     Log::info("Invalid document uploaded, exiting ...");
                        //     $scan->status = "REJECTED";
                        //     $scan->save();
                        //     $this->ready = true;
                        //     // return ['success' => false, "error" => "Invalid document uploaded ..."];
                        //     // exit();
                        //     throw new Exception('Invalid document type uploaded, exiting ...');
                        // }
                        // Log::debug($statement);
                        // exit();

                        $mtnMomoProcessor = new Processor($statement);

                        // send Heartbeat
                        // $this->heartbeat();

                        $mtnMomoProcessor->initializeStatement();

                        // if (is_array($mtnMomoProcessor->validateOwnershipByName($ownerInfo["name"]))) {
                        //     // Unauthorized action, names mismatch
                        //     $scan->validation_failure = true;
                        //     $scan->validation_error = $mtnMomoProcessor->validateOwnershipByName($ownerInfo["name"]["error"]);
                        // }

                        // Save Statement
                        // $statement;
                        // $saveStatementResponse = $mtnMomoProcessor->persistStatement($ownerInfo, $scanId, $auth);
                        // if (is_string($saveStatementResponse)) {
                        //     $results["persisted"] += 1;
                        //     // Update scan object with statement id as object id
                        //     $scan->object_id = $saveStatementResponse;
                        //     $scan->save();

                        //     // Categorize and persist transactions
                        //     $mtnMomoProcessor->categorizeAndPersistTransactions($auth, $saveStatementResponse, $ownerInfo["id"]);
                        // } else if ($saveStatementResponse) {
                        //     $results["existing"] = $results["existing"] + 1;
                        // }
                        // capture insights
                        $insights = $mtnMomoProcessor->captureInsights();
                        // prepare response
                        $response = $mtnMomoProcessor->prepareResponse($insights, $includeRecords, false);
                        // $results['insights'][] = $insights;
                        $this->ready = true;
                        $results['workflowId'] = $workflowId;
                        $results['data'][] = $insights;
                        // $results['data'][] = $statement;
                        $results['transactionCount'][] = $statement->getTotalTransactions();
                        $results['basic_information'][] = $response['basic_information'];
                        $results['account_information'][] = $response['account_information'];

                        $results["successful"] = $results["successful"] + 1;
                        $results["persisted"] = $results["persisted"] + 1;
                        $scan->total_pages = intval($response['account_information']['total_pages']);
                        $scan->data = $response;
                        $charges = ChargesHelper::getCost(intval($scan->total_pages));
                        Log::debug("Charges here is : " . $charges);
                        $scan->service_charge = $charges;
                        $scan->normalized = $response['transactions']['normalized'];
                        $scan->save();

                        
                    } catch (PDOException $e) {
                        Log::error($e->getMessage());
                        $results["error_count"] = $results["error_count"] + 1;
                        $results["errors"][] = ["error" => $e->getMessage(), "error_code" => "001"];
                    } catch (Exception $e) {
                        Log::error($e->getMessage());
                        $results["error_count"] = $results["error_count"] + 1;
                        $results["errors"][] = ["error" => $e->getMessage(), "error_code" => "001"];
                    }

                    //         break;

                    //     case "ATGH":
                    //         // Process Airtel Tigo Gh Statement 
                    //         break;
                    // endswitch;
                }
                break;
            case "BANKSTATEMENT":
                // scan bank statement(s)
                break;

            case "INVOICE":
                // scan invoice(s)
                break;

            case "RECEIPT":
                // scan receipt(s)
                break;

        endswitch;


        return $results;
    }
}

<?php

namespace App\Activities;

use App\Models\Scan;
use Workflow\Activity;
use Log;

class EnrichActivity extends Activity
{
    // public $tries = 1;

    // Define the activity to run
    public function execute(string $scanId)
    {
        
        $result = [];

        // Log::debug($parsedData);

        // Update scan record, ensure its   idempotent
        $scan = Scan::findOrFail($scanId);

        $results = [
            'error_count' => 0,
            'recordUpdated' => false,
            'successful' => false,
        ];

        // Only update status if its Scanning
        if ($scan->status === 'SCANNING') {
            $scan->status = 'ENRICHING';
            $scan->save();
            $results['recordUpdated'] = true;
            $scan->status = 'READY';
            $scan->save();
        }

        // $workflow = WorkflowStub::load($workflowId);

        // $ready = $workflow->getReady();

        $results['successful'] = true;

        return $result;
    }
}

.

Also, I will like to know what is the best way to exit from an activity execution, do you throw an exception?
Thanks in advance.

1 reply

rmcdaniel May 9, 2025
Maintainer

Best way is to throw a non-retryable exception.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Workflow is trigger infintely #176

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Uh oh!

Workflow is trigger infintely #176

Uh oh!

Uh oh!

demobiel Apr 26, 2024

Replies: 2 comments · 2 replies

Uh oh!

rmcdaniel Apr 26, 2024 Maintainer

Uh oh!

demobiel Apr 27, 2024 Author

Uh oh!

Uh oh!

jeanlinux Mar 3, 2025

Uh oh!

rmcdaniel May 9, 2025 Maintainer

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

demobiel
Apr 26, 2024

Replies: 2 comments 2 replies

rmcdaniel
Apr 26, 2024
Maintainer

demobiel Apr 27, 2024
Author

jeanlinux
Mar 3, 2025

rmcdaniel May 9, 2025
Maintainer