Skip to content

MultiCurl: Callbacks aren't called and new connections aren't started until a batch completes #903

Open
@ricnava00

Description

@ricnava00

When starting a MultiCurl batch with >1 parallel connections, in the default case no handling actions are taken (executing callbacks or starting new connections) until a whole parallel batch is concluded
This is because of the loop at

do {
$status = curl_multi_exec($this->multiCurl, $active);
} while ($status === CURLM_CALL_MULTI_PERFORM);
that doesn't finish until all the requests in the batch are completed

This doesn't happen when calling setRequestTimeAccuracy() before starting the MultiCurl, since the other branch is executed.

Since i couldn't trigger the curl_multi_exec error described in the previous lines, I don't know what can be changed and what cannot, so I'm writing this issue instead.

Here is a simple test to replicate the issue:

use Curl\MultiCurl;
const TEST_URL="https://ash-speed.hetzner.com/100MB.bin";
$multi_curl = new MultiCurl();
//$multi_curl->setRequestTimeAccuracy(); //Uncomment for expected output

$start_time=microtime(true);
foreach(range(1,12) as $i){
	$ch=$multi_curl->addGet(TEST_URL);
	$ch->setOpt(CURLOPT_MAX_RECV_SPEED_LARGE, 100*$i*1024);
	$ch->setRange("0-".(500*1024));
	$ch->beforeSend(function() use ($i, $start_time){
		echo "Start $i at ".round(microtime(true)-$start_time,2)."\n";
	});
	$ch->complete(function() use ($i, $start_time){
		echo "End $i at ".round(microtime(true)-$start_time,2)."\n";
	});
}

$multi_curl->setConcurrency(3);
$multi_curl->start();

This downloads the first 500KB of a test file multiple times, at speeds increasing by 100KB/s for every connection, and with 3 parallel connections
This means that connection 1 should take 500/100 = 5 seconds, connection 2 should take 500/200 = 2.5 seconds, and so on

Without calling setRequestTimeAccuracy, this is the output I get

Start 1 at 0
Start 2 at 0
Start 3 at 0
End 3 at 4.96
End 2 at 4.96
End 1 at 4.97
Start 4 at 4.97
Start 5 at 4.97
Start 6 at 4.97
End 6 at 6.21
End 5 at 6.21
End 4 at 6.22
Start 7 at 6.22
Start 8 at 6.22
Start 9 at 6.22
End 9 at 6.93
End 8 at 6.93
End 7 at 6.93
Start 10 at 6.93
Start 11 at 6.93
Start 12 at 6.93
End 12 at 7.43
End 11 at 7.44
End 10 at 7.44

As you can see, while connection 3 is supposed to be 3 times faster, it will wait until connection 1 and 2 are finished, and then connections 4, 5 and 6 will be started

When calling setRequestTimeAccuracy, the output becomes

Start 1 at 0
Start 2 at 0
Start 3 at 0
End 3 at 1.66
Start 4 at 1.66
End 2 at 2.48
Start 5 at 2.48
End 4 at 2.9
Start 6 at 2.9
End 5 at 3.48
Start 7 at 3.48
End 6 at 3.73
Start 8 at 3.73
End 7 at 4.19
Start 9 at 4.19
End 8 at 4.35
Start 10 at 4.35
End 9 at 4.74
Start 11 at 4.74
End 10 at 4.85
Start 12 at 4.85
End 1 at 4.96
End 11 at 5.19
End 12 at 5.26

Now, the callback for connection 3 is called as expected, and connection 4 is started right after
The total runtime is also lower, since new connections are started as soon as possible

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy