Repository navigation
Can't stop circular dataflow blocks #612
Description
I'm trying to implement a crawler with System.Threading.Tasks.Dataflow, the core feature is to have a TransformBlock, which accepts an url as input, request the url, parse the response, and find more urls for the same block.
Then I noticed a problem that I cannot complete/stop this pipeline. The following code simulates my problem:
using System;
using System.Threading;
using System.Threading.Tasks.Dataflow;
namespace ConsoleApp1
{
class Program
{
static void Main()
{
int count = 0;
var block = new TransformBlock<string, string>(uri =>
{
//await Task.Delay(10);
Console.WriteLine($"Downloading {count}");
if (Interlocked.Increment(ref count) > 10)
{
return null;
}
return uri;
});
var linkOptions = new DataflowLinkOptions { PropagateCompletion = false };
block.LinkTo(block, linkOptions, (x) => !string.IsNullOrEmpty(x));
block.Post("");
block.Post("");
while (true)
{
Console.WriteLine($"Check Count: {count}");
if (count >= 2)
{
Console.WriteLine("Complete");
block.Complete();
break;
}
Thread.Sleep(20);
}
Console.WriteLine("After Check");
block.Completion.Wait();
Console.WriteLine("Done");
}
}
}The output is:
Check Count: 0
Downloading 0
Downloading 1
Check Count: 2
Complete
After Check
And program stuck here, block.Completion.Wait() never return.
I've also tried to have two blocks link to each other, yet the result is same.
So what can I do about my code? How can I stop this dataflow?
Activity
ZhaoXiangXML commentedon Dec 6, 2019
Versions of things: Windows 10.0.17763.864 64bit, .NET Core 3.0, System.Threading.Tasks.Dataflow 4.11.0
ZhaoXiangXML commentedon Dec 6, 2019
And I've also tried
PropagateCompletion = true, the result is all sameZhaoXiangXML commentedon Dec 6, 2019
Well problem solved by adding
block.LinkTo(DataflowBlock.NullTarget<string>(), linkOptions);, thanks to the help of the sample code