Skip to content

Forward Socket

Introduction

The forward socket is a new feature added to StreamPU to improve the performance and the flexibility in some applications. As mentioned in the Socket section, the SFWD works as an input and output at the same time. It receives its dataptr from the input bound socket and this same pointer is sent to all the output bound sockets, which means that all the consecutive tasks bound by SFWD share the same memory space.

graph LR;
A(FWD)-->B(FWD); A(FWD)-.->K{MEM};
B(FWD)-->C(FWD); B(FWD)-.->K{MEM};
C(FWD)-->F(FWD); C(FWD)-.->K{MEM};
F(FWD)-.->K{MEM};

Technical Improvements

The implementation of the forward socket for sequences was mainly straightforward because it behaves the same way as the input and output sockets. We just had to distinguish when it is used as an input and when it is used as an output. However, the most challenging part was to combine forward socket with the pipeline. Especially when forward sockets are bound from one stage to an other.

Forward Sockets and Pipelines

As explained in the Adaptor section, a pool of buffers is used between each stage of the pipeline. The adaptor gets a buffer from this pool and uses it to update the output socket of its pull task (dataptr attribute). This output socket is then bound to the input socket of the next tasks. In other words, all the input sockets connected to the pull output socket need to be updated with the new dataptr address.

The forward sockets are all pointing to the same dataptr, so getting a new buffer means that we have to update the dataptr of all the consecutive bound forward sockets to this new memory space. In the sequence, the same update needs to be done in the reversed way when the dataptr is exchanged at the end of the stage. For that, we added two recursive methods as explained in the sequence section (see explore_thread_rec() and explore_thread_rec_reverse() methods).

Moreover, when multiple forward sockets are crossing pipeline stages, we need to check if these forward sockets are pointing to a same dataptr or to different dataptrs. In the case where multiple forward sockets are crossing pipeline stages and are pointing to the same dataptr, only one buffer need to be created in the adaptors. To detect this, a map to record the previous dataptr is used (see the fwd_source variable in the code).

Tests

Some specific tests have been added to the project to validate the robustness of the forward socket implementation.

Specific for Forward Socket

double chain
test-pipeline-double-chain.
test-pipeline-double-chain -t 3
The purpose of this graph is to test the buffer exchange with SIO and SFWD, both on the same stage.

forward inter stage
test-complex-pipeline-full-fwd.
test-complex-pipeline-full-fwd -t 3
The purpose of this graph is to test a SFWD bound to two SFWD in two different stages, and how the buffer exchange behaves with connections between distant stages \(S1\) and \(S4\).

forward inter stage
test-complex-pipeline-mix-fwd.
test-complex-pipeline-mix-fwd -t 3
The purpose of this graph is to test a SFWD bound to three SFWD in three different stages (\(S1 \rightarrow S2\), \(S1 \rightarrow S4\) and \(S1 \rightarrow S5\)), and how the buffer exchange behaves with connections between distant stages. Additionally, a traditional relay task (\(t4\) with an input and an output socket) has been added in stage \(S2\). The \(t7\) compare task ensures that all the 3 SFWD have the same contents. It is expected that the final values in \(t8\) are \(init + 2\) and the final values in \(t9\) are \(init + 1\). In this test, task to task binding is used to ensure that \(t4\) is executed before \(t6\) and \(t8\) is executed before \(t9\) (see the oriented dashed lines).

double inter stage
test-complex-pipeline-inter-stage.
test-complex-pipeline-inter-stage -t 3
This test is a combination of the previous tests, we have a SOUT bound to a SIN in stage \(S2\) and a SFWD in stage \(S4\).