Jekyll2017-11-02T23:06:02+00:00/This isn’t a place you’re looking forSome random ideas about programming and related stuffAmazon ALB for websockets and what’s wrong with it2017-11-01T23:40:42+00:002017-11-01T23:40:42+00:00/aws/alb/2017/11/02/hateful-amazon-application-load-balancer<p>That’s very annoying, but it seems that despite the fact ALB (Amazon year old “new” load balancer) declares websocket support, it’s not very good idea to use it for long-living connections!</p>That’s very annoying, but it seems that despite the fact ALB (Amazon year old “new” load balancer) declares websocket support, it’s not very good idea to use it for long-living connections!Thoughts on C++ async API2017-10-14T10:40:42+00:002017-10-14T10:40:42+00:00/c++/async/2017/10/14/cpp-async-promises<h2 id="preface">Preface</h2>
<p>Nowadays everything needs to be asynchronous. We all know that. No one suddenly believes that having a dedicated thread for every task is a good idea anymore.
And developing C++ library every programmer has to solve the problem: how to provide library user good way to track and control application state</p>
<h2 id="good-old-times">Good old times</h2>
<p>Classical C++ way to handle asynchronous operation is to provide an object which implements some listener interface.
This approach works well for years and generations of C++ (and other languages) programmers.
But downsides ot this method are obvious after first glance at the code: it’s extremely wordy:
For every possible listener customer of your library should create class implementing listener interface and deal with it’s lifetime.
If your class has a several listeners, you’re doomed.</p>
<p>Let’s consider communication library, which creates persistent connection and allows user to send and receive messages.
So we need (at least) three listeners:</p>
<ul>
<li>connection state listener</li>
<li>incoming message listener</li>
<li>outgoing message listener</li>
</ul>
<p>Which means that “classical” flavor of this interface looks like that</p>
<figure class="highlight"><pre><code class="language-c--" data-lang="c++"><span class="cm">/**
* Listener to track message state
*/</span>
<span class="k">class</span> <span class="nc">IMessageStateListener</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="k">virtual</span> <span class="kt">void</span> <span class="n">onMessageSent</span><span class="p">()</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">virtual</span> <span class="kt">void</span> <span class="n">onMessageFailed</span><span class="p">()</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">virtual</span> <span class="kt">void</span> <span class="n">onMessageCancelled</span><span class="p">()</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">};</span>
<span class="cm">/**
* Library itself
*/</span>
<span class="k">class</span> <span class="nc">ILibrary</span>
<span class="p">{</span>
<span class="n">pubic</span><span class="o">:</span>
<span class="cm">/**
* Every call of sendMessage starts an async operation, so it needs dedicated listener instance.
* Also it needs a possibilty to cancel this operation
*/</span>
<span class="n">CancellationToken</span> <span class="n">sendMessage</span><span class="p">(</span><span class="n">Message</span> <span class="n">message</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">shared_ptr</span><span class="o"><</span><span class="n">IMessageStateListener</span><span class="o">></span> <span class="n">messageStateListener</span><span class="p">);</span>
<span class="kt">void</span> <span class="n">cancelMessage</span><span class="p">(</span><span class="n">CancellationToken</span> <span class="n">canellationToken</span><span class="p">);</span>
<span class="p">};</span></code></pre></figure>
<p>It looks not that bad, and quite easy to implement for library developer. But it involves quite a lot of work from library user.
And this is what we need to fix in order to provide easier interface and free up application developer for some actual work, not countless listeners and event routing.</p>
<h2 id="javascript-fanboy-enters">JavaScript fanboy enters</h2>
<p>At this point I have to confess. During last two years I was dealing a lot with javascript. I know, this language is not really well respected in C++ community, but it is all about asynchronousity. So it definitely has something to learn.
And nowadays the most adopted approach of handling asynchronous events in JS is Promise. Yes, they have added async/await, and it is really awesome, but let’s keep this theme for another blog post.
So when I got to the problem of building robuts async interface, I evaluated options, and decided to go the way of promises, as most straightforward and easy to implement option.</p>
<p>Here the question arises: we do have std::promise right now. What is the problem then?
Unfortunately, std::promise/future are lacking few features:</p>
<ul>
<li>continuations and chaining. Some of these already are in experimental, but you don’t want to expose “experimental” in production api, do you?</li>
<li>it lacks proper control over thread model which library uses to resolve these promises (can be solved)</li>
<li>it doesn’t support cancellation</li>
</ul>
<p>So yes, I did it. I wrote my own promises library.
Though I’m not selling this library here. This article is more about approach, not the particular implementation.</p>
<p>At this point let’s see how our interface may look like using promises:</p>
<figure class="highlight"><pre><code class="language-c--" data-lang="c++"><span class="k">class</span> <span class="nc">ILibrary</span>
<span class="p">{</span>
<span class="k">public</span><span class="o">:</span>
<span class="n">cancellable_future</span><span class="o"><</span><span class="n">Result</span><span class="o">></span> <span class="n">sendMessage</span><span class="p">(</span><span class="n">Message</span> <span class="n">message</span><span class="p">);</span>
<span class="p">};</span></code></pre></figure>
<p>This interface is quite short and self-exlanatory. But this is not only advantage.
We also do not need to track observer lifetime, we do not need to store canellation token, we don’t need to pass cancellation token and method which consumes it to every place where we may want to trigger a cancellation.</p>
<figure class="highlight"><pre><code class="language-c--" data-lang="c++"><span class="c1">//Creating a future instance
</span><span class="k">auto</span> <span class="n">future</span> <span class="o">=</span> <span class="n">libraryInstance</span>
<span class="p">.</span><span class="n">sendMessage</span><span class="p">({</span> <span class="s">"What a lovely day "</span> <span class="p">})</span>
<span class="p">.</span><span class="n">then</span><span class="p">([](</span><span class="k">auto</span> <span class="n">response</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">code</span> <span class="o">==</span> <span class="mi">200</span><span class="p">)</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Message has been delivered: "</span> <span class="o"><<</span> <span class="n">response</span><span class="p">.</span><span class="n">code</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Message has not been delivered: "</span> <span class="o"><<</span> <span class="n">response</span><span class="p">.</span><span class="n">code</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">});</span></code></pre></figure>
<p>And remember, you now have a uniformed interface to cancel async operation:</p>
<figure class="highlight"><pre><code class="language-c--" data-lang="c++"><span class="c1">// And remember, you can do cancel everywhere!
</span><span class="n">future</span><span class="p">.</span><span class="n">cancel</span><span class="p">();</span></code></pre></figure>
<p>Also, promises are chainable:</p>
<figure class="highlight"><pre><code class="language-c--" data-lang="c++"><span class="k">auto</span> <span class="n">future</span> <span class="o">=</span> <span class="n">libraryInstance</span>
<span class="p">.</span><span class="n">sendMessage</span><span class="p">({</span> <span class="s">"What a lovely day "</span> <span class="p">})</span>
<span class="p">.</span><span class="n">then</span><span class="p">([](</span><span class="k">auto</span> <span class="n">response</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="n">code</span> <span class="o">==</span> <span class="mi">200</span><span class="p">)</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Message has been delivered: "</span> <span class="o"><<</span> <span class="n">response</span><span class="p">.</span><span class="n">code</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Message has not been delivered: "</span> <span class="o"><<</span> <span class="n">response</span><span class="p">.</span><span class="n">code</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">})</span>
<span class="p">.</span><span class="n">then</span><span class="p">([]()</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"I'm a chained method!"</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="n">promise</span><span class="o"><</span><span class="kt">int</span><span class="o">></span> <span class="n">p</span><span class="p">;</span>
<span class="c1">// and make sure this promise will be resolved somewhere else
</span>
<span class="k">return</span> <span class="n">p</span><span class="p">.</span><span class="n">get_cancellable_future</span><span class="p">([]()</span> <span class="p">{});</span> <span class="c1">// I'm to lazy to write cancellaton here
</span> <span class="p">})</span>
<span class="p">.</span><span class="n">then</span><span class="p">([](</span><span class="kt">int</span> <span class="n">answer</span><span class="p">)</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"I'm asynchronously chained and answer is "</span> <span class="o"><<</span> <span class="n">answer</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">});</span></code></pre></figure>
<h2 id="thread-model">Thread model</h2>
<p>What is the different in javascript and c++ runtimes? Everything! But one of the most important things - c++ programs usually have multiple threads.
Which means that unlike JS when you just put whatever you want in “then” section, you need to care about non-blocking behaviour, thread safety, etc.
This is the thing which we should care about, since we have a continuation. Imagine, library user has long operation inside the continuation, and we’re calling it from some core library thread.
Such situation would cause whole library to stuck until continuation execution finished.
This is not a kind of behaviour.</p>
<p>Most straightforward approach would be to resolve promise (call set_value in case of c++ promise) in a thread which you want this callback to be called from. This means manual rescheduling the event on some dedicated thread every time we need to resolve the promise.
That is quite possible, but again, involves some manual work inside the library. At least we need to have this dedicated thread, easy way to reschedule event, etc.
Sounds boring.</p>
<p>One of the options is to bundle this functionality inside the promises library.
Futures Java 8 has something like that. In every “thenContinueAsync” or smth like that user can pass a thread pool, which is going to be used to execute continuation method.
This definetely should work for c++ also.
Though it means that it’s a library user who has control over thread model. That’s also good thing, but not good enough.
Let’s modify this approach a bit. We can still support passing the thread pool to every “then” call, but we also need a possibility to pass it to promise constructor. Or “get_future” method.</p>
<figure class="highlight"><pre><code class="language-c--" data-lang="c++"><span class="n">promise</span><span class="o"><</span><span class="kt">void</span><span class="o">></span> <span class="n">p</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">future</span> <span class="o">=</span> <span class="n">p</span><span class="p">.</span><span class="n">get_future</span><span class="p">(</span><span class="n">executor</span><span class="p">);</span>
<span class="n">future</span><span class="p">.</span><span class="n">then</span><span class="p">([]()</span> <span class="p">{</span>
<span class="n">std</span><span class="o">::</span><span class="n">cout</span> <span class="o"><<</span> <span class="s">"Supplied executor decided which thread I'm executed on!"</span> <span class="o"><<</span> <span class="n">std</span><span class="o">::</span><span class="n">endl</span><span class="p">;</span>
<span class="p">});</span></code></pre></figure>
<p>Where executor can be really anything.
For example, we can implement “synchronous executor”, which just calls the continuaton on the same thread
We can implement “single thread executor”, which guarantees that all callback are being schedulet on the same thread
And we can implement “multiple thread executor”, which just takes threads from threadpool</p>
<p>This method provides enough flexibility and helps to concentrate the knowledge about thread model at the place where promise is created.</p>
<h2 id="real-life-experience">Real life experience</h2>
<p>This was a real life problem I had refactoring a c++ library.
What can I say:</p>
<ul>
<li>using such approach I got much simplier interface</li>
<li>users of library were happy enough since they don’t need to create a gazillion of listeners and track them manually</li>
<li>continuation got simpler</li>
<li>thread model became much more uniform</li>
</ul>
<p>So, don’t hesitate, use promises in c++ and be happy!</p>
<h2 id="alternatives">Alternatives</h2>
<p>There are few other options to facilitate asynchronous operations:</p>
<ul>
<li>RxCpp and Observables</li>
<li>Coroutines</li>
<li>Probably much more options I’m not aware of</li>
</ul>
<p>I won’t pretend I’ve tried them all.
Coroutines and async await looked too much pain to build it in to the existing application. Probably works better when you writing app from scratch.
Observables are cool for series of events. E.g. it, probably, would be better option for incoming messages.</p>
<p>So promises is simple and robust solution. As every tool it shouldn’t be considered as silver bullet. It won’t solve all your problems. But if you use it with care, it is extremely useful.</p>Preface