Discussion:
[boost] [Beast] Questions Before Review
Artyom Beilis via Boost
2017-06-25 21:02:13 UTC
Permalink
Hello Vinnie Falco,

I have several design questions regarding this library before the review.

DISCLIMER:

My name is Artyom Beilis.
I'm the author of CppCMS C++ Web Framework: http://cppcms.com
It may seems like it can be in some competition with
Beast but I don't see it this way as Beast is too low level library that
does not addresses typical problems of developers needing Web API/Site
running using C++.

CppCMS addresses totally different problems in different way.


So there are my design related questions:


Header Only Design
---------------------------

As somebody who worked with guys who do web development, it was very
clear that compilation time is an issue.

A simple build of http_server_small.cpp example requires about 6s(!)
of compilation time g++ 5.4
http_server_fast.cpp takes 7s

In comparison entire message board containing templates code flow SQL and more
leads to only 4s - non paralel build.
(https://github.com/artyom-beilis/cppcms/tree/master/examples/message_board
)

I understand that you work on base of Boost.Asio - which itself has
unacceptable compilation times but having for web
application development may be serious deal breaker for big projects.

Which Leads me to another design Choice


Template Meta-programming Based Design instead of classic Object Oriented Design
----------------------------------------------------------------------------------------------------------------
I see that almost every object there is some kind of template object.

It almost mimics the API of Boost.Asio which on its way has its limitations.
I remember I needed to implement FastCGI over TCP and Unix domain sockets
virtually leading to same code besides some initial bind - and it required to
create the entire parser using template programming because the socket itself
was template object different for UNIX and TCP.

I have strong feeling that lots of stuff can actually be done using classic OOP.
Can I implement same server code for both SSL and non SSL HTTP protocol
without use of templates in the class itself?
And my current final question

Who are the potential users of the library?
-------------------------------------------------------


If I need to send a simple http request to server it does not seem
to do simple enough job (see your FAQ) so I'll probably be better
with feature rich libcurl.

If I need to implement complex web server/service I don't have
cookies, sessions, forms, html?

If I need to implement RESTful API do I have all stuff like URL mapping of
different paths to callbacks, protocol abstraction http/https
(basic stuff - I don't see it there correct me if I wrong)

Is it for web server developers? Maybe I'd be happier with simple
HTTP protocol parser that I can integrate to any socket API?

Can you please give more accurate description of who is this
library intended for?


Best Regards,
Artyom Beilis

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-25 21:32:57 UTC
Permalink
On Sun, Jun 25, 2017 at 2:02 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
It may seems like it can be in some competition with
Beast but I don't see it this way as Beast is too low level library that
does not addresses typical problems of developers needing Web API/Site
running using C++.
Beast is not a competitor to CppCMS, as the former is a simple
protocol layering on top of Asio while the latter is a Web Development
Framework. Beast does not intend to be a web framework now or ever.
Post by Artyom Beilis via Boost
I understand that you work on base of Boost.Asio - which itself has
unacceptable compilation times but having for web
application development may be serious deal breaker for big projects.
I'll note that the very first complaint during the Boost.Http review
(2015) was that the library is not header-only. If Asio compilation
times are unacceptable then you will find Beast compilation times
unacceptable as well, since Beast is very much tied to Boost.Asio as a
base layer.
Post by Artyom Beilis via Boost
I see that almost every object there is some kind of template object.
It almost mimics the API of Boost.Asio which on its way has its limitations.
Very astute of you to notice! In fact, an overarching design goal of
Beast is "emulate Asio in all possible ways." If you feel that
Boost.Asio has limitations, you find similar or identical limitations
in Beast. Conversely, those who feel that Boost.Asio has tremendous
flexibility and expressive power, will also find similar in Beast. I
find myself in the latter camp, but your mileage may vary.
Post by Artyom Beilis via Boost
I have strong feeling that lots of stuff can actually be done using classic OOP.
There is no question that it would be possible to create non-template
classes for every conceivable choice of template parameters but I'm
not sure that's a productive use of time and while it might satisfy
most users to have explicit instantiations for
boost::asio::ip::tcp::socket and
boost::asio::ssl::stream<boost::asio::ip::tcp::socket> it is certain
to leave at least a few people in the lurch. For example, "that guy"
who wants to write an AsyncStream wrapper for his libuv socket so it
can work with Beast.
Post by Artyom Beilis via Boost
Can I implement same server code for both SSL and non SSL HTTP protocol
without use of templates in the class itself?
If you can do it with Asio then you can do it with Beast. Typically
this is done with type erasure for the main template parameters which
are 1. the stream object, 2. the buffer sequence, and 3. the
completion handler. You can do the same with Beast, except that you
will also need to make a commitment to one type for requests and one
type for responses. This makes some of Beast's cool template driven
features not work but it seems like you're okay with that tradeoff.
Post by Artyom Beilis via Boost
Who are the potential users of the library?
Beast is primarily aimed at library writers, although it frequently
satisfies high level application developers even with its verbose
interfaces since the alternatives are objectively worse.
Post by Artyom Beilis via Boost
If I need to send a simple http request to server it does not seem
to do simple enough job (see your FAQ) so I'll probably be better
with feature rich libcurl.
Beast is definitely not a replacement for libcurl, and likely will
never be. Libcurl is a full featured HTTP client, while Beast is
merely an HTTP protocol layering on top of TCP/IP using Boost.Asio's
asynchronous model.

However, someone eventually is going to make a better libcurl by
writing it on top of Beast, and then we'll see some nice things like
C++ native interfaces, with lambdas, type safety, perhaps template
arugments defining HTTP client specific concepts such as
BasicAuthenticator or MultiPartConsumer (I made those up but the names
allude to their usage). Most importantly, someone who writes a C++
libcurl on top of Beast will be able to present complete, type-safe,
fully-intact message objects to the caller when asked, and then that
message object can be passed to other libraries or algorithms built
with Beast.

There's a network effect when you have a good set of vocabulary types
and algorithms, and Beast aims to provide that for HTTP (and
WebSocket!). Just like how the standard library gives us a common
voice with which to explain solutions to general computer science
problems, Beast provides the pieces for building solutions to HTTP and
networking problems. My goal is to guide Beast through Boost
acceptance, widespread adoption, and then a proposal for the C++
Standard Library.
Post by Artyom Beilis via Boost
If I need to implement complex web server/service I don't have
cookies, sessions, forms, html?
...
If I need to implement RESTful API do I have all stuff like URL mapping of
different paths to callbacks, protocol abstraction http/https
(basic stuff - I don't see it there correct me if I wrong)
Everything you described is out of scope for Beast. I'm hopeful that
the existence of Beast will inspire programmers to create new higher
level libraries to solve the problems you listed in a composable and
interoperable way.
Post by Artyom Beilis via Boost
Is it for web server developers?
You can build a web server with Beast. But as you pointed out, you
will have to either write for yourself or find in other libraries some
important pieces like a URI parser, cookies, authentication, codecs
for the various Content-Encoding, a multi-part/mime library, and so on
and so forth.

Obviously we would all be happy if these things existed already with
perfect C++ friendly interfaces and with composability and
interoperability in mind but they don't. Beast is the first down
payment on that dream.
Post by Artyom Beilis via Boost
Maybe I'd be happier with simple
HTTP protocol parser that I can integrate to any socket API?
And you can do that with Beast, the beast::http::basic_parser is
designed for users to derive from if they want to take advantage of
the parser implementation. You can derive your own parser and feed the
buffers from wherever you want. Or if you like Beast's message model
(I happen to think its rather well designed but that's just my
opinion) you can use the beast::http::parser by feeding in buffers to
produce messages which you can do whatever you want with. This is
covered in the documentation, with examples:
http://vinniefalco.github.io/beast/beast/using_http/buffer_oriented_parsing.html

If you want to serialize HTTP messages to buffers without using Asio
and sockets you can do that too, using beast::http::serializer, also
covered in the documentation, with examples:
http://vinniefalco.github.io/beast/beast/using_http/buffer_oriented_serializing.html
Post by Artyom Beilis via Boost
Can you please give more accurate description of who is this
library intended for?
Its a fair question although I thought the Introduction in the
documentation gave a pretty good idea. Its sort of a hard question to
answer because really we have never seen a library like this before
which is so low level so I think people just don't know what to think
of it especially when they start comparing it to other libraries or
applications whose descriptions include the word "HTTP"

The library is intended for anyone who wants to build something
resembling a high level library that exists today, such as Boost.Http,
cpprestsdk, cppnetlib, CppCMS, or any of those libraries which
implement either an HTTP client or an HTTP server. Instead of
reinventing the HTTP protocol code from scratch Beast gives you a
running start. It takes care of all the boring details of reading and
writing HTTP messages so that you can write your library at a higher
level.

Any library that is refactored to use Beast will immediately see
benefits. It will result in less code in the target library. That
library's interfaces will become more composable. You can take message
objects produced by one library written with Beast and pass them into
algorithms which take message objects that reside in other libraries
written with Beast.

So to answer your question, Beast is aimed at anyone who is writing
software that wants to speak the HTTP (or WebSocket!) protocols using
Boost.Asio and eventually the Networking-TS
(http://cplusplus.github.io/networking-ts/draft.pdf), which is a
polished version of Boost.Asio that will certainly become part of the
C++ standard library.

I will conclude by saying that eventually, only networking libraries
which are coded against Boost.Asio, the Networking-TS, or ultimately
the C++ Standard Library version on or after which the Networking-TS
has been integrated, can possibly be taken seriously. After all, who
would want to use a library that doesn't use the "standard"
implementation of networking?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Artyom Beilis via Boost
2017-06-27 20:30:46 UTC
Permalink
On Mon, Jun 26, 2017 at 12:32 AM, Vinnie Falco via Boost
Post by Vinnie Falco via Boost
On Sun, Jun 25, 2017 at 2:02 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
I understand that you work on base of Boost.Asio - which itself has
unacceptable compilation times but having for web
application development may be serious deal breaker for big projects.
I'll note that the very first complaint during the Boost.Http review
(2015) was that the library is not header-only.
Actually I was one of the guys who told that header only version
is bad. I still consider it a huge design flaw.
Post by Vinnie Falco via Boost
Post by Artyom Beilis via Boost
Can I implement same server code for both SSL and non SSL HTTP protocol
without use of templates in the class itself?
If you can do it with Asio then you can do it with Beast.
I didn't see how to do it with Asio.
Post by Vinnie Falco via Boost
Typically
this is done with type erasure for the main template parameters which
are 1. the stream object, 2. the buffer sequence, and 3. the
completion handler. You can do the same with Beast, except that you
will also need to make a commitment to one type for requests and one
type for responses. This makes some of Beast's cool template driven
features not work but it seems like you're okay with that tradeoff.
Can you provide an example?
Post by Vinnie Falco via Boost
Post by Artyom Beilis via Boost
Who are the potential users of the library?
Beast is primarily aimed at library writers, although it frequently
satisfies high level application developers even with its verbose
interfaces since the alternatives are objectively worse.
Can you give examples of high level application users?
It was pointed that ripple uses beast. What it used for?
What is implemented using Beast?

I looked to API with thoughts of implementing some basic
things that used by http clients/servers (Very basic)...

- How do I get the query string (the thing after "?" in URL)
- Content type parsing? (see for example [1])
- Cookie - I found a constant representing a header... i.e. parsing on my.
- multipart-form data parsing? Generation? urlencode/decode?

I understand not to have high level form handling (validation, setting)
but being able to parse urlencoded string or get cookies as key/pair values
is sort of very basic thing.

If it is barely HTTP headers parser what do I need it for? Parsing
HTTP headers is quite trivial stuff. The tricky one of understanding
what is send and how to handle it.


Additional Points
---------------------

Looking over examples I noticed that the library user needs to manage
all timeouts/keep alive connections. Am I right?
Post by Vinnie Falco via Boost
Post by Artyom Beilis via Boost
If I need to send a simple http request to server it does not seem
to do simple enough job (see your FAQ) so I'll probably be better
with feature rich libcurl.
Beast is definitely not a replacement for libcurl, and likely will
never be.
[snip]
However, someone eventually is going to make a better libcurl by
writing it on top of Beast,
Or wrapping excellent well supported and debugged library
with good C++ interface. BTW I used libcurl a lot and it
was and is the Wwiss army knife in all related to URL requests.
Post by Vinnie Falco via Boost
Post by Artyom Beilis via Boost
If I need to implement complex web server/service I don't have
cookies, sessions, forms, html?
...
If I need to implement RESTful API do I have all stuff like URL mapping of
different paths to callbacks, protocol abstraction http/https
(basic stuff - I don't see it there correct me if I wrong)
Everything you described is out of scope for Beast. I'm hopeful that
the existence of Beast will inspire programmers to create new higher
level libraries to solve the problems you listed in a composable and
interoperable way.
Question?

Why should I choose to write something over beast
when there quite a lot of good solutions providing much
more (higher level interface)

I mean it feels like addresses very small niche of users that would
most likely write their own HTTP parsers that would exactly
fit their needs.
Post by Vinnie Falco via Boost
Post by Artyom Beilis via Boost
Is it for web server developers?
You can build a web server with Beast. But as you pointed out, you
will have to either write for yourself or find in other libraries some
important pieces like a URI parser, cookies, authentication, codecs
for the various Content-Encoding, a multi-part/mime library, and so on
and so forth.
Which basically the 99% of entire work that need to be done...
Post by Vinnie Falco via Boost
Obviously we would all be happy if these things existed already with
perfect C++ friendly interfaces and with composability and
interoperability in mind but they don't. Beast is the first down
payment on that dream.
Actually there are lots of...

Client side: curl/curl++, poco, QHttp
Server side: CppCMS. Wt, tntnet and more

What is important they provide way more useful stuff
besides being integrated to Boost.Asio.

I'm not telling this to discourage you I'm telling it
because Beast looks to me more like a tiny layer
above socket or an assembly language for HTTP
handling.
Post by Vinnie Falco via Boost
Post by Artyom Beilis via Boost
Can you please give more accurate description of who is this
library intended for?
Its a fair question although I thought the Introduction in the
documentation gave a pretty good idea. Its sort of a hard question to
answer because really we have never seen a library like this before
which is so low level so I think people just don't know what to think
of it especially when they start comparing it to other libraries or
applications whose descriptions include the word "HTTP"
The library is intended for anyone who wants to build something
resembling a high level library that exists today, such as Boost.Http,
cpprestsdk, cppnetlib, CppCMS, or any of those libraries which
implement either an HTTP client or an HTTP server. Instead of
reinventing the HTTP protocol code from scratch Beast gives you a
running start. It takes care of all the boring details of reading and
writing HTTP messages so that you can write your library at a higher
level.
As somebody who actually wrote http parser and lots of boring
details the HTTP parser was the _simplest and easiest_ thing
to implement. What I had to work hard on it?

- How to manage application life time, timeouts, errors, sync/async
io,
- How to handle huge contents in secure way (consider uploading 10GB file)
- How to parse cookies, content type etc in safe way.

And I'm not talking about basic higher level stuff like forms CSRF, security
sessions etc.

I mean HTTP itself was tiny part of the problem, the hard problems
that may interest the library/framework develop are out of Beast scope.

Regards,

Artyom Beilis



[1] http://cppcms.com/cppcms_ref/latest/classcppcms_1_1http_1_1content__type.html

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-27 21:28:12 UTC
Permalink
On Tue, Jun 27, 2017 at 1:30 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
It was pointed that ripple uses beast. What it used for?
What is implemented using Beast?
Ripple implements an HTTP and WebSocket service which uses a JSON
interface to interact with the Ripple network (a decentralized ledger
with a built-in cryptocurrency). [1]
Post by Artyom Beilis via Boost
If it is barely HTTP headers parser what do I need it for?
Parsing HTTP messages is just one great feature of Beast. Other great
features include:

* An HTTP message model
* Read HTTP message to stream synchronously
* Read HTTP message to stream asynchronously
* Write HTTP message to stream synchronously
* Write HTTP message to stream asynchronously
* Built-in chunked encoding codec
* Obtain a serialized buffer representation of an HTTP message
* Produce an HTTP message from serialized buffers
* A full WebSocket implementation
* Classes for implementing composed operations
* Adapters for working with buffer sequences
* Metafunctions to determine if a type meets a concept
* New models of DynamicBuffer
* static_string

Some users have a need to create an HTTP message and send it, or to
receive an HTTP message and process it. Sometimes it is in the context
of a web server or general HTTP client but not always. Often they want
to use WebSocket.

A programmer who just wants to send and receive an HTTP message from
within their code has to do what exactly? Link with the CppCMS
framework? Include a copy of cUrl in their app? They can use Beast for
that. And they *are* using Beast for that, the library is steadily
growing in users and popularity.

In particular, Beast's message model is quite robust with lots of
customization points. Its been designed to avoid the flaws exhibited
in other libraries [2]
Post by Artyom Beilis via Boost
Looking over examples I noticed that the library user needs to manage
all timeouts/keep alive connections. Am I right?
Yes, and this is also stated in the documentation. With a big red
WARNING sign [3]
Post by Artyom Beilis via Boost
Why should I choose to write something over beast
when there quite a lot of good solutions providing much
more (higher level interface)
That's a great question. There are quite a few Beast users now, the
GitHub issues have been lively and I have received many emails of
thanks (and even more emails asking for help!). Apparently some people
are finding Beast quite useful. If you think that there are other
solutions which are better suited for your purpose, I am not
surprised, Beast was not written to solve all problems. Just to handle
the cases for which it was written.

One area where Beast has a likely advantage, is that it is more suited
for standardization (inclusion in the C++ standard library) than any
other library. For exactly the reason underlying your questions - it
doesn't try to do too much. But what it does do, it does correctly.
More specifically I am making this claim:

* Beast demonstrates what a standardized, low-level implementation of
HTTP should look like

To put it bluntly, any program which is not sending and receiving HTTP
messages in a fashion similar to Beast (and eventually, using Beast
explicitly) is likely doing it wrong, to the same extent that someone
who insists on implementing their own version of common objects found
in the standard library are doing it wrong (granted there are some
special cases).
Post by Artyom Beilis via Boost
I'm not telling this to discourage you I'm telling it because Beast looks to me more like
a tiny layer above socket or an assembly language for HTTP handling.
Yes!

Here comes the shocker...quite a few people prefer Beast over other
libraries judging from the activity in the repository and feedback
I've received. Because it doesn't impose odd choices on its users.

Its exactly what you said, a thin protocol layering on top of Asio.
Beast implements just enough of the HTTP protocol to get messages in
and out and leaves the "odd choices" up to users. I think that's how
it should be for a low level library. And I think that C++ and Boost
in particular, needs this low level component.
Post by Artyom Beilis via Boost
And I'm not talking about basic higher level stuff like forms CSRF, security
sessions etc.
I mean HTTP itself was tiny part of the problem, the hard problems
that may interest the library/framework develop are out of Beast scope.
Well, I don't know that I agree with that. One of Beast's most
important features is how closely it tracks Asio interfaces and best
practices. I have gotten feedback from users that this is one of the
greatest strengths. Of course, if someone is allergic to Asio then
they will similarly be allergic to Beast.

To say that HTTP itself was a "tiny part of the problem" is I think to
disregard the enormous effort and problem solving that went into
getting Beast to where it is. Even if I were to agree and say it was a
tiny part of the problem, it is a part that has not been satisfied to
a thorough degree and of a quality sufficient to have a chance at
Boost acceptance or C++ standardization.

...and it has WebSockets :)

Thanks

[1] https://github.com/ripple/rippled/tree/7b0d48281049c3fec7fafcb7ce5cea045367ae1f/src/ripple/server/impl

[2] http://vinniefalco.github.io/beast/beast/design_choices/http_comparison_to_other_librari.html

[2] http://vinniefalco.github.io/beast/beast/using_networking/asio_refresher.html

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-28 15:25:27 UTC
Permalink
On Tue, Jun 27, 2017 at 1:30 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
Can you give examples of high level application users?
...
Or wrapping excellent well supported and debugged library
with good C++ interface. BTW I used libcurl a lot and it
was and is the Wwiss army knife in all related to URL requests.
I asked a user this question:

"Is Beast otherwise living up to your expectations / fulfilling your needs?"

His reply [1]

"absolutely, I have implemented embedded http servers across a whole
suite of programs (still need to update using the new framework) and I
was able to exorcise curl for simple http GET/POST requests with a
short extension/asyncification of the http client example. Had to do
async because there's essentially no support for timeouts in asio sync
methods (see this awesome wontfix -
https://svn.boost.org/trac10/ticket/2832). so I recommend a note about
this somewhere and maybe an async example or a link to some asio async
resources. I can try to sanitize /release my extension if that would
help

getting rid of curl on an embedded target takes my runtime link
dependencies down from 50-something to ~17, it's crazy. curl is
extremely hard to statically link on debian and very much embraces the
"unix way" of having tons of small supporting elements. so for a
couple GET/POST it gets to be a bit much. most competitive libraries
are similarly huge and can be hard to set up

so, anyway, this fills a big void in simple to set up and non-bloated
http tools for c++, thank you"

[1] https://github.com/vinniefalco/Beast/issues/548#issuecomment-311690875

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Niall Douglas via Boost
2017-06-25 21:36:05 UTC
Permalink
Post by Artyom Beilis via Boost
Template Meta-programming Based Design instead of classic Object Oriented Design
----------------------------------------------------------------------------------------------------------------
I see that almost every object there is some kind of template object.
It almost mimics the API of Boost.Asio which on its way has its limitations.
I remember I needed to implement FastCGI over TCP and Unix domain sockets
virtually leading to same code besides some initial bind - and it required to
create the entire parser using template programming because the socket itself
was template object different for UNIX and TCP.
I have strong feeling that lots of stuff can actually be done using classic OOP.
This is by far one of the most annoying things I find with the ASIO API
design, and it's one of my strongest regrets that that design choice is
entering the standard. I can see why it seemed like a good idea back in
the day, but it's an obvious API design mistake now.

Thanks to my experience with ASIO, both AFIO v1 and v2 avoid the
templating of objects and APIs like the plague. It's totally unnecessary
in C++ 11, and perhaps even in C++ 98.

Where you need to consume or produce arbitrary scatter-gather buffers,
use gsl::span<T> or equivalent. AFIO v2 has a really simple,
non-template, yet just as powerful scatter-gather buffer design as ASIO
yet without using a single template. It's all thanks to gsl::span<T>
which takes care of the container adaptation for you.

I'm very proud of that part of AFIO v2, and I'd strongly recommend all
new code needing scatter-gather buffers copy that approach rather than
copying ASIO, whose scatter-gather buffer support is way over wraught
and over engineered for what is needed.

The other reason why ASIO probably felt it needed to template everything
is for arbitrary user supplied completion handlers. Yet in AFIO v2 I've
reduced that to a single common shared template which (cleverly if I do
say so myself) performs the sole memory allocation anywhere in AFIO v2
where we allocate the platform dependent storage for the async
scatter-gather i/o alongside storage for the completion handler in a
single malloc() call. This is done at the front of the async API so the
rest of the API implementation then works with that type erased
instance, allowing it to live in a precompiled DLL, and eliminating
templates for almost all of the implementation.

Now I haven't looked at BEAST yet, and BEAST is not ASIO, it's not as
low level, and it may need to work with custom strings, or
discontinuously stored lumps of HTTP data, so I certainly can see that
char iterators are going to get used a lot, and those imply templates.
That said if everything is templatised I would struggle to see the
motivation. We'll see when the review begins.

BTW Vinnie I assume BEAST works just fine with non-header ASIO, it might
be worth doing a quick benchmark to see how much of your compile times
are ASIO and how much BEAST. I would imagine it will come up.

Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-25 21:48:49 UTC
Permalink
On Sun, Jun 25, 2017 at 2:36 PM, Niall Douglas via Boost
Post by Niall Douglas via Boost
Where you need to consume or produce arbitrary scatter-gather buffers,
use gsl::span<T> or equivalent.
As gsl::span<> is only capable of representing ranges of elements
contiguous in memory, it is not capable of representing the majority
of Beast's buffer sequences. Examples:

https://github.com/vinniefalco/Beast/blob/8982e14aa65b9922ac5a00e5a5196a08dfa8f29e/include/beast/http/impl/fields.ipp#L103

https://github.com/vinniefalco/Beast/blob/8982e14aa65b9922ac5a00e5a5196a08dfa8f29e/include/beast/core/buffer_cat.hpp#L17

https://github.com/vinniefalco/Beast/blob/8982e14aa65b9922ac5a00e5a5196a08dfa8f29e/include/beast/core/impl/multi_buffer.ipp#L121

Using gsl::span<> effectively type-erases the buffer sequence which is
inefficient, since it must be copied to a vector equivalent in order
to be used.
Post by Niall Douglas via Boost
I certainly can see that char iterators are going to get used a lot
Beast never templates on the character type. All operations on
structured data (for example HTTP headers) are performed on a single
contiguous memory buffer of chars. This is done for performance
reasons.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Niall Douglas via Boost
2017-06-25 23:42:57 UTC
Permalink
Post by Vinnie Falco via Boost
On Sun, Jun 25, 2017 at 2:36 PM, Niall Douglas via Boost
Post by Niall Douglas via Boost
Where you need to consume or produce arbitrary scatter-gather buffers,
use gsl::span<T> or equivalent.
As gsl::span<> is only capable of representing ranges of elements
contiguous in memory, it is not capable of representing the majority
Think gsl::span<gsl::span<char>>.

Actually AFIO v2 uses gsl::span<std::pair<char *, size_t>>, but that's
to avoid the bounds check per buffer that gsl may do. In your case that
bounds check won't be as important, so gsl::span<gsl::span<char>> makes
lots of sense. Not least for all the extra automated static analysis you
get for free.
Post by Vinnie Falco via Boost
Post by Niall Douglas via Boost
I certainly can see that char iterators are going to get used a lot
Beast never templates on the character type. All operations on
structured data (for example HTTP headers) are performed on a single
contiguous memory buffer of chars. This is done for performance
reasons.
Hmm. I find that highly surprising. I'd have thought that as the HTTP
data comes in - in chunks - your code would work with discontiguous storage.

Are you memory copying instead?

Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-25 23:54:46 UTC
Permalink
On Sun, Jun 25, 2017 at 4:42 PM, Niall Douglas via Boost
Post by Vinnie Falco via Boost
Using gsl::span<> effectively type-erases the buffer sequence which is
inefficient, since it must be copied to a vector equivalent in order
to be used.
Think gsl::span<gsl::span<char>>.
gsl::span<gsl::span<char>> effectively requires the callers to set
aside storage for an array of std::pair<void const*, std::size_t>. The
Asio equivalent is std::vector<boost::asio::const_buffer> which as you
can see is inefficient as std::vector is not cheap to copy.

Asio's ConstBufferSequence concept of course includes
gsl::span<gsl::span<char>> and std::vector<boost::asio::const_buffer>
as models but it also includes the types I provided which cannot be
modeled using gsl::span without intermediate storage. If you feel this
is an acceptable tradeoff for your library, I am quite happy for you
but I'll stick with the use of templates and the ConstBufferSequence
concept since that is what is on track for standardization.
Post by Vinnie Falco via Boost
I'd have thought that as the HTTP comes in - in chunks -
your code would work with discontiguous storage.
I thought that too and built quite a bit on top of a parser that
worked in chunks, until I used a profiler and did some comparisons to
other implementations...
Post by Vinnie Falco via Boost
Are you memory copying instead?
...when I discovered it is actually much, much faster to memcpy
discontiguous buffers into a new, allocated buffer so that the parser
can assume the entire structured portion is in a single "span" (to use
the terminology you prefer). And of course if the caller already has
the data in a single buffer then there's no need to memcpy. The class
beast::flat_buffer is provided for that purpose.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Niall Douglas via Boost
2017-06-26 02:12:25 UTC
Permalink
Post by Vinnie Falco via Boost
On Sun, Jun 25, 2017 at 4:42 PM, Niall Douglas via Boost
Post by Vinnie Falco via Boost
Using gsl::span<> effectively type-erases the buffer sequence which is
inefficient, since it must be copied to a vector equivalent in order
to be used.
Think gsl::span<gsl::span<char>>.
gsl::span<gsl::span<char>> effectively requires the callers to set
aside storage for an array of std::pair<void const*, std::size_t>. The
Asio equivalent is std::vector<boost::asio::const_buffer> which as you
can see is inefficient as std::vector is not cheap to copy.
Asio's ConstBufferSequence concept of course includes
gsl::span<gsl::span<char>> and std::vector<boost::asio::const_buffer>
as models but it also includes the types I provided which cannot be
modeled using gsl::span without intermediate storage. If you feel this
is an acceptable tradeoff for your library, I am quite happy for you
but I'll stick with the use of templates and the ConstBufferSequence
concept since that is what is on track for standardization.
AFIO v2 doesn't allocate memory except exactly once per async i/o
initiated. So the scatter-gather buffer list is given to the OS
immediately, and therefore no copies of that list are needed as the OS
takes them. Most users I should imagine would therefore build
scatter-gather lists on the stack as they'll be thrown away immediately,
indeed I usually feed it curly braced initializer_lists personally,
least typing.

In BEAST's case, I can see that scatter-gather buffers may need to be
kept for longer. Except ...
Post by Vinnie Falco via Boost
Post by Vinnie Falco via Boost
I'd have thought that as the HTTP comes in - in chunks -
your code would work with discontiguous storage.
I thought that too and built quite a bit on top of a parser that
worked in chunks, until I used a profiler and did some comparisons to
other implementations...
Post by Vinnie Falco via Boost
Are you memory copying instead?
...when I discovered it is actually much, much faster to memcpy
discontiguous buffers into a new, allocated buffer so that the parser
can assume the entire structured portion is in a single "span" (to use
the terminology you prefer). And of course if the caller already has
the data in a single buffer then there's no need to memcpy. The class
beast::flat_buffer is provided for that purpose.
... you're using memcpy to append incoming chunks of TCP data anyway,
obviating the need for scatter-gather.

Now I personally find it quite surprising that you found that memory
copying is faster than a high quality discontinuous storage iterator
implementation with ring buffered page aligned DMA friendly TCP i/o.
Anyone who has ever claimed that to me before, when I examined their
code I found they'd been using a naive RandomAccessIterator
implementation which was doing two pointer indirections per access,
which is obviously going to be not quick. What they should have done was
implement BidirectionalIterator only, and had it cache when it next
needs to do a lookup into the next discontiguous section, thus becoming
very fast indeed because we're not trashing the CPU caches and not
stalling the out of order execution. This is the kind of stuff you need
if you want to get a single CPU core feeding a >= 10 Gbps NIC.

The question then becomes is how expensive is losing random access on
your parser implementation? Cloning these BidirectionalIterators is
extremely quick (three pointers each), plus parsing HTTP tends to work
in offsets anyway. Any XML parser I've ever written I keep a stack of
where I am in the tag hierarchy as I run forwards - so I would reckon a
ForwardIterator would actually be enough in practice.

But I'll grant you that HTML is not XML, it's much harder to parse tag
soup. So you may be right. Nevertheless I think you are going to have to
back up your claim that memory copying all incoming data is faster
rather than being bad implementation techniques with discontinuous
storage, as surely I am not alone here in being surprised at such a claim.

Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-26 02:28:23 UTC
Permalink
On Sun, Jun 25, 2017 at 7:12 PM, Niall Douglas via Boost
Post by Niall Douglas via Boost
Most users I should imagine would therefore build
scatter-gather lists on the stack as they'll be thrown away immediately,
indeed I usually feed it curly braced initializer_lists personally,
Thus imposing limitation on the size of the buffer sequence.
Post by Niall Douglas via Boost
I think you are going to have to back up your claim that memory
copying all incoming data is faster rather than being bad implementation
techniques with discontinuous storage
I'll let Kazuho back it up for me since I use his ideas:
https://github.com/h2o/picohttpparser

Here's the slide show explaining the techniques:
https://www.slideshare.net/kazuho/h2o-20141103pptx

And here is an example of the optimizations possible with linear
buffers, which I plan to directly incorporate into Beast in the near
future:
https://github.com/h2o/picohttpparser/blob/2a16b2365ba30b13c218d15ed9991576358a6337/picohttpparser.c#L110

Of course if you think you can do better I would love to see your
working parser that operates on multiple discontiguous high quality
ring buffered page aligned DMA friendly storage iterators so that I
might compare the performance. The good news is that you can do so
while leveraging Beast's message model to produce objects that people
using Beast already understand. Except that you'll be producing them
much, much faster (which is a win-win for everyone).

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Niall Douglas via Boost
2017-06-26 14:47:16 UTC
Permalink
Post by Vinnie Falco via Boost
Post by Niall Douglas via Boost
Most users I should imagine would therefore build
scatter-gather lists on the stack as they'll be thrown away immediately,
indeed I usually feed it curly braced initializer_lists personally,
Thus imposing limitation on the size of the buffer sequence.
The kernel imposes very significant limits on the size of the buffer
list anyway: some OSs as low as 16 scatter-gather buffers per i/o, and
as low as 1024 scatter-gather buffers in flight across the entire OS. So
when you initiate an async i/o, you may get a resource temporarily
unavailable error for even a single buffer, let alone two.

On top of that, even if the OS accepts more, the DMA hardware has a
fixed size buffer list capacity. 64 is not uncommon, and that's after
the kernel has split your virtual memory scatter-gather list into
physical memory plus added its own scatter-gather headers. So 32 buffers
is a very realistic limit, and 16 is the portable maximum.

(AFIO v2 doesn't involve itself whatsoever with any of this, it sends
what you ask for to the OS, and reports back whatever errors the OS does)
Post by Vinnie Falco via Boost
Post by Niall Douglas via Boost
I think you are going to have to back up your claim that memory
copying all incoming data is faster rather than being bad implementation
techniques with discontinuous storage
https://github.com/h2o/picohttpparser
https://www.slideshare.net/kazuho/h2o-20141103pptx
And here is an example of the optimizations possible with linear
buffers, which I plan to directly incorporate into Beast in the near
https://github.com/h2o/picohttpparser/blob/2a16b2365ba30b13c218d15ed9991576358a6337/picohttpparser.c#L110
Ah, I see you're referring to SIMD. I thought you were claiming that
linear buffer based parsers were significantly faster than forward only
iterator based parsers.

You solve that problem by doing all i/o in multiples of the SIMD length,
and memcpy any tail partial SIMD length at the end of a partial i/o into
the next buffer. This avoids memory copying, yet keeps SIMD.
Post by Vinnie Falco via Boost
Of course if you think you can do better I would love to see your
working parser that operates on multiple discontiguous high quality
ring buffered page aligned DMA friendly storage iterators so that I
might compare the performance. The good news is that you can do so
while leveraging Beast's message model to produce objects that people
using Beast already understand. Except that you'll be producing them
much, much faster (which is a win-win for everyone).
You're the person bringing the library before review, not me. If you
have a severe algorithmic flaw in your implementation, reviewers would
be right to reject your library. They did so with me for AFIO v1, so it
was on me to start AFIO again from scratch.

Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-26 15:46:43 UTC
Permalink
On Mon, Jun 26, 2017 at 7:47 AM, Niall Douglas via Boost
If you have a severe algorithmic flaw in your implementation, reviewers
would be right to reject your library.
If you are stating that Beast has a "severe algorithmic flaw" then
please back up your claims with more than opinion. However, note the
following:

* At the time Boost.Http was reviewed it used the NodeJS parser which
operates in chunks [1]. No "severe algorithmic flaw" came up then.

* PicoHTTPParser, which Beast's parser is based on, outperforms NodeJS
by over 600% [2]

* For parsers operating on discontiguous buffers, structured elements
such as request-target, field names, and field values must be
flattened (linearized) to be presented to the next layer which means
temporary storage and buffer copying [3, 4]. So buffer copies cannot
be avoided. Beast makes the decision to do one big buffer copy up
front instead of many small buffer copies as it goes. The evidence
shows this tradeoff is advantageous.

But maybe you are suggesting that functions like basic_fields::insert
should take as their first parameter `gsl::span<gsl::span<char>>`
instead of `string_view` [5]? That would be quite inconvenient.

[1] https://github.com/nodejs/http-parser

[2] https://github.com/fukamachi/fast-http/tree/6b9110347c7a3407310c08979aefd65078518478

[3] https://github.com/vinniefalco/Beast/blob/8982e14aa65b9922ac5a00e5a5196a08dfa8f29e/test/benchmarks/nodejs_parser.hpp#L664

[4] "In case you parse HTTP message in chunks...your data callbacks
may be called more than once"
https://github.com/nodejs/http-parser/blob/master/README.md#callbacks

[5] https://github.com/vinniefalco/Beast/blob/8982e14aa65b9922ac5a00e5a5196a08dfa8f29e/include/beast/http/fields.hpp#L365

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Niall Douglas via Boost
2017-06-26 16:15:33 UTC
Permalink
Post by Vinnie Falco via Boost
On Mon, Jun 26, 2017 at 7:47 AM, Niall Douglas via Boost
If you have a severe algorithmic flaw in your implementation, reviewers
would be right to reject your library.
If you are stating that Beast has a "severe algorithmic flaw" then
please back up your claims with more than opinion.
I did not state that. I did indicate surprise at the choice to use
memcpy over other available techniques.
Post by Vinnie Falco via Boost
* At the time Boost.Http was reviewed it used the NodeJS parser which
operates in chunks [1]. No "severe algorithmic flaw" came up then.
I think I was the only person to recommend acceptance in that review? I
remember people had problems with the API design and intent. So people
may not have bothered thinking about implementation too much if the API
design was a fail.
Post by Vinnie Falco via Boost
* PicoHTTPParser, which Beast's parser is based on, outperforms NodeJS
by over 600% [2]
If used as its author intended. I didn't get the impression the author
expected to you memcpy its input.
Post by Vinnie Falco via Boost
* For parsers operating on discontiguous buffers, structured elements
such as request-target, field names, and field values must be
flattened (linearized) to be presented to the next layer which means
temporary storage and buffer copying [3, 4].
Why?

LLVM doesn't do this. So why should Beast?
Post by Vinnie Falco via Boost
So buffer copies cannot
be avoided. Beast makes the decision to do one big buffer copy up
front instead of many small buffer copies as it goes. The evidence
shows this tradeoff is advantageous.
It may be relative to the other things you've tested against. But it may
not be correct.

For example, another zero copy DMA friendly trick is to do all socket
i/o via a file so the kernel can DMA directly to/from the kernel file
cache. From your perspective, you simply need to map a view onto that
kernel file cache into your process, and indeed you can get your
contiguous span of chars this way without using memcpy by doing a
rolling mmap() of the span you need to see.

However, there are costs to that approach. TLB shootdown can be a real
problem, though less than some often think. Your TCP window size needs
to be big and your NIC not a toy NIC for it to be worthwhile. Most
consumer PCs are fitted with toy NICs, so any benchmarking done on those
won't be realistic. You also need to bypass ASIO and go direct, because
ASIO's async scatter-gather buffer implementation is crap
(https://github.com/chriskohlhoff/asio/issues/203).

Now it may be that reviewers feel that the memory copy is not
significant relative to the rest of Beast. I may feel this way too after
I've studied the code, and for example I'd likely accept memcpy of the
HTTP request and response headers but not accept memcpy of the message
body. We'll see when the review begins.

Niall
--
ned Productions Limited Consulting
http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-26 16:25:11 UTC
Permalink
On Mon, Jun 26, 2017 at 9:15 AM, Niall Douglas via Boost
Post by Niall Douglas via Boost
Post by Vinnie Falco via Boost
* PicoHTTPParser, which Beast's parser is based on, outperforms NodeJS
by over 600% [2]
If used as its author intended. I didn't get the impression the author
expected to you memcpy its input.
phr_parse_request() requires the entire header in a single contiguous
memory buffer [1]
Post by Niall Douglas via Boost
Why?
...
LLVM doesn't do this. So why should Beast?
Answered already [2]

[1] https://github.com/h2o/picohttpparser/blob/2a16b2365ba30b13c218d15ed9991576358a6337/picohttpparser.h#L51

[2] http://boost.2283326.n4.nabble.com/Beast-Questions-Before-Review-tp4696141p4696160.html

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-26 16:26:52 UTC
Permalink
On Mon, Jun 26, 2017 at 9:15 AM, Niall Douglas via Boost
Post by Vinnie Falco via Boost
Post by Niall Douglas via Boost
Why?
...
LLVM doesn't do this. So why should Beast?
Answered already [2]
Apologies, footnote 2 link was incorrect, this is the proper link:

[2] http://boost.2283326.n4.nabble.com/Beast-Questions-Before-Review-td4696141.html#a4696163

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-27 02:56:08 UTC
Permalink
On Mon, Jun 26, 2017 at 9:15 AM, Niall Douglas via Boost
Post by Niall Douglas via Boost
LLVM doesn't do this
Link to source code, please?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert Ramey via Boost
2017-06-25 22:02:46 UTC
Permalink
Post by Artyom Beilis via Boost
Hello Vinnie Falco,
I have several design questions regarding this library before the review.
Header Only Design
---------------------------
As somebody who worked with guys who do web development, it was very
clear that compilation time is an issue.
A simple build of http_server_small.cpp example requires about 6s(!)
of compilation time g++ 5.4
http_server_fast.cpp takes 7s
In comparison entire message board containing templates code flow SQL and more
leads to only 4s - non paralel build.
(https://github.com/artyom-beilis/cppcms/tree/master/examples/message_board
)
I'm certainly missing something here. Why is compile time of 4-7
seconds any kind of issue? When would a web developer compile the
library. Please expand on this as I'm not getting it all.

Robert Ramey


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Robert via Boost
2017-06-30 18:53:03 UTC
Permalink
Post by Robert Ramey via Boost
Post by Artyom Beilis via Boost
Hello Vinnie Falco,
I have several design questions regarding this library before the review.
Header Only Design
---------------------------
As somebody who worked with guys who do web development, it was very
clear that compilation time is an issue.
A simple build of http_server_small.cpp example requires about 6s(!)
of compilation time g++ 5.4
http_server_fast.cpp takes 7s
In comparison entire message board containing templates code flow SQL and more
leads to only 4s - non paralel build.
(https://github.com/artyom-beilis/cppcms/tree/master/examples/message_board
)
I'm certainly missing something here. Why is compile time of 4-7
seconds any kind of issue? When would a web developer compile the
library. Please expand on this as I'm not getting it all.
Robert Ramey
I too am missing something regarding compilation time. A C# asp.net web
site takes roughly five seconds to, "Build." Then launching it within
Visual Studio adds more time, coming up to around one minute and
sometimes more, until the browser completes rendering the page.

Robert
Post by Robert Ramey via Boost
_______________________________________________
http://lists.boost.org/mailman/listinfo.cgi/boost
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Artyom Beilis via Boost
2017-06-30 20:21:43 UTC
Permalink
Post by Artyom Beilis via Boost
Hello Vinnie Falco,
I have several design questions regarding this library before the review.
Header Only Design
---------------------------
As somebody who worked with guys who do web development, it was very
clear that compilation time is an issue.
A simple build of http_server_small.cpp example requires about 6s(!)
of compilation time g++ 5.4
http_server_fast.cpp takes 7s
In comparison entire message board containing templates code flow SQL and more
leads to only 4s - non paralel build.
(https://github.com/artyom-beilis/cppcms/tree/master/example
s/message_board
)
I'm certainly missing something here. Why is compile time of 4-7 seconds
any kind of issue? When would a web developer compile the library. Please
expand on this as I'm not getting it all.
Robert Ramey
I too am missing something regarding compilation time. A C# asp.net web
site takes roughly five seconds to, "Build." Then launching it within
Visual Studio adds more time, coming up to around one minute and sometimes
more, until the browser completes rendering the page.

Robert



You aren't compared to asp.net but to php/python/rails/nodejs that are the
trendiest web development tools today.

Even big projects start quite quickly with java / servlet

And as a reference one of the simple examples provided compiles for about
30s ! If you multiply this by complexity of normal project the times become
intolerable.

Considering that beast does 1/100 of what others tools provide (even if in
cool asio way) the price becomes too high.

All it needed is to do classic OOD instead and you'll get much wider
audience. With much better error reporting by the compiler, smaller code,
and easier development, consider streight forward support of HTTP / https
out of the box in some code.

Doing temple based solution asio style for HTTP library IMHO is a design
error.

Artyom

_______________________________________________
Unsubscribe & other changes:
Robert Ramey via Boost
2017-07-01 04:53:14 UTC
Permalink
Post by Artyom Beilis via Boost
You aren't compared to asp.net but to php/python/rails/nodejs that are the
trendiest web development tools today.
Even big projects start quite quickly with java / servlet
And as a reference one of the simple examples provided compiles for about
30s ! If you multiply this by complexity of normal project the times become
intolerable.
Considering that beast does 1/100 of what others tools provide (even if in
cool asio way) the price becomes too high.
All it needed is to do classic OOD instead and you'll get much wider
audience. With much better error reporting by the compiler, smaller code,
and easier development, consider streight forward support of HTTP / https
out of the box in some code.
Doing temple based solution asio style for HTTP library IMHO is a design
error.
Artyom
I'm sorry, I'm just not seeing the relevance of compile time to anything
here.


Robert Ramey

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Artyom Beilis via Boost
2017-07-01 19:27:03 UTC
Permalink
http vs WebSocket
--------------------------

I noticed some strange difference in these requests :

// Wrap the now-connected socket in a websocket stream
websocket::stream<tcp::socket&> ws{sock};

// Perform the websocket handhskae
ws.handshake(host, "/", ec);
if(ec)
return fail("handshake", ec);



This one is fast and easy to use does all it needed

Now HTTP is very different story

http::request<http::string_body> req;
req.method(http::verb::get);
req.target("/");
req.version = 11;
req.set(http::field::host, host + ":" +
std::to_string(sock.remote_endpoint().port()));
req.set(http::field::user_agent, BEAST_VERSION_STRING);
req.prepare_payload();

Most of stuff can be defined there by default: version 1.1, Host header -
required by http/1.1 why should I bother? User agent put one by default
unless user wants - you do all this stuff for WS handshake so why not
there? Why so complicated?

-----------------

Protocol Premier HTTP/1.1 example without host. Your example isn't valid
HTTP /1.1 request.

HTTP /1.1 unlike 1.0 requires host header

Artyom

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-07-01 19:49:30 UTC
Permalink
On Sat, Jul 1, 2017 at 12:27 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
http vs WebSocket
...
...
ws.handshake(host, "/", ec);
...
This one is fast and easy to use does all it needed
Yes. There is a bit of a difference between the WebSocket interface
versus HTTP. The scope of HTTP is limited to being a low level
protocol layering. However in the case of WebSocket, rfc6455 is
nowhere near as complex as rfc7230 (the HTTP specification). You might
find that WebSocket feels more "complete."

Even so, users have the ability to customize the WebSocket handshake
by using a decorator:

http://vinniefalco.github.io/beast/beast/ref/beast__websocket__stream/handshake_ex/overload3.html

This allows the User-Agent to be set. beast::websocket::stream only
uses a default User-Agent when the decorator does not set one, or if
the decorator is absent. So there is no performance penalty of setting
it versus not setting it.

In any event, I'm glad to see WebSocket getting some attention :)
Post by Artyom Beilis via Boost
Now HTTP is very different story
...
Most of stuff can be defined there by default: version 1.1
beast::http::message defaults to 1.1 instead of using an uninitialized
`int`. The example sets the version explicitly for expository
purposes. It is not a goal of Beast to have a compact user-facing
interface for building HTTP requests.
Post by Artyom Beilis via Boost
Host header - required by http/1.1 why should I bother?
Beast doesn't have the information needed to fill out this field. But
even if it did, it should not be defaulted because...
Post by Artyom Beilis via Boost
User agent put one by default unless user wants - you do all this stuff for WS handshake
so why not there? Why so complicated?
Beast doesn't make odd choices on behalf of the user, its a low level
library. If you feel that Beast is too complicated, you can write your
own layer above it which makes choices for you that you think are
appropriate. If enough people agree with your choices, perhaps you
could publish it as a higher level library written on top of Beast!

That is my goal, for there to be competition in the marketplace of
ideas of what choices a higher level library might make (and such a
library should be written on top of Beast of course).

Even if we wanted the behavior you describe it would require that a
beast::http::message using basic_fields should allocate memory in its
constructor to set a field value. Beast follows the "pay for what you
use" principle. Users have shown overwhelming support for Beast's
style of not making these decisions on behalf of users.

Furthermore, the Beast message container can use any instance of the
Fields concept [1], including those which don't support insertion of
arbitrary strings. What would we do in that case?

Thanks

[1] http://vinniefalco.github.io/beast/beast/concept/Fields.html

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Artyom Beilis via Boost
2017-07-01 20:29:02 UTC
Permalink
In any event, I'm glad to see WebSocket getting some attention :)



I assume it is because despite the hipe around WS they are far less useful
in reality for one simple reason - they aren't HTTP.

I consider server sent events as far better solution for server side push
from both client and server side, see 1 below.

WS virtually never work with proxies well and require https to work
properly. Especially today when SSL inspection becoming common WS are
quite problematic.


On the bright side it looks like WS part of beast is far more useful than
HTTP one.


Artyom Beilis


1: blog.cppcms.com/post/107

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-07-01 20:34:42 UTC
Permalink
On Sat, Jul 1, 2017 at 1:29 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
1: blog.cppcms.com/post/107
https://www.html5rocks.com/en/tutorials/eventsource/basics/

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-07-01 20:36:11 UTC
Permalink
On Sat, Jul 1, 2017 at 1:29 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
1: blog.cppcms.com/post/107
https://www.html5rocks.com/en/tutorials/eventsource/basics/

Message was cut off, sorry about that.

Anyway, Beast is not limited to just web servers serving web content.
WebSockets are very useful when you want a two-way channel such as for
games. I'll note you did not address that use-case in your CppCMS blog
post and intstead listed the common example of stock quotes, which are
one-way.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Vinnie Falco via Boost
2017-06-30 18:09:01 UTC
Permalink
On Sun, Jun 25, 2017 at 2:02 PM, Artyom Beilis via Boost
Post by Artyom Beilis via Boost
Can you please give more accurate description of who is this
library intended for?
This user seems to like it:


"I must say that I love the simplicity and small size of Beast. Even I,
a novice C++ programmer, have been able to get it setup and use it."

https://www.reddit.com/r/cpp/comments/6k4xnj/beast_formal_review_branch_frozen_starts_july_1/djm117f/

Thanks

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Loading...