* SOCKS handshake handler implementation
* Soundness
* Remove placeholder text
* Sad path tests
* Soundness
* Docs
* Fix workflow tests
* Make handler removable
* Protect methods
* Prevent test crashes
* Cleanup public types
* Add test that writing after auth fails
* Add force handler removal tests
* Remove client and server state from public api
* Explicitly handle states
* Remove promises
* Fix test
* Add data to authentication complete
* Refactor to add authentication complete flag
Implement a SOCKSv5 client according to RFC 1928. Server implementation will be added in another PR to keep sizes down.
https://datatracker.ietf.org/doc/html/rfc1928
A few meaningful changes:
Add all relevant types used across SOCKS clients and servers
Add a state machine used to manage a connection from a clients side
Add a channel handler that should be added at the very start of a channel pipeline
* fix crash in LengthFieldBasedFrameDecoder for malicious length values
Motivation:
LengthFieldBasedFrameDecoder will cause a fatal error if the length value does not fit into an `Int`.
This can happen if `lengthFieldLength` is set to `.eight` and we are on a 64 bit platform or if `lengthFieldLength` is set to `.four` and we are on a 32-bit platform.
If we then receive a length field value which is greater than `Int.max` the conversion from `UInt` to `Int` will cause a fatal error.
This could be abused to crash a server by only sending 4 or 8 bytes.
Modifications:
safely convert UInt64 & UInt32 to Int and throw an error if they can't be represented as an Int
Result:
- LengthFieldBasedFrameDecoder with lengthFieldLength set to `.eight` can no longer crash the server on a 64-bit platform
- LengthFieldBasedFrameDecoder with lengthFieldLength set to `.four` can no longer crash the server on a 32-bit platform
* use early exit instead of XCTSkipIf
* add support for `.eight` on 32-bit platforms
* limit frame length to `Int32.max`
* change test names
* throw correct error
* fix compilation for Swift 5.0 and add NIO prefix to error enum
* add test for maximum allowed length and one above the maximum allowed length
Signed-off-by: David Nadoba <dnadoba@gmail.com>
* run XCTest script
Signed-off-by: David Nadoba <dnadoba@gmail.com>
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Motivation:
The RSocket protocol uses a 24 bit length field
Modifications:
- add two new methods readInteger and writeInteger on ByteBuffer that support reading and writing integers of any size.
- add a new case (.three) to ByteLength
Result:
LengthFieldBasedFrameDecoder & LengthFieldPrepender do now support a 24 bit length field
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Motivation:
As a rule of thumb we should always forward channel events to the next
handler. #106 added an implementation for `channelInactive` but forgot
to forward it.
Modifications:
- forward `channelInactive` in the `RequestResponseHandler`
Result:
Handlers after the `RequestResponseHandler` will recieve
`channelInactive`.
Motivation:
It's possible for channels to be closed without an error; and the
`RequestResponseHandler` should tolerate that by failing any promises
for which it does not have a response for.
Modifications:
- Add `ClosedBeforeReceivingResponseError`
- Fail outstanding promises with `ClosedBeforeReceivingResponseError` in
`RequestResponseHandler.channelInactive`
- Add a test.
Result:
Outstanding request promises are failed when the channel becomes inactive.
Motivation:
It's useful to know the overhead we could be adding by including
the PCAP handler.
Modifications:
Add a new executable based on the NIO performance testing executable.
Result:
There is a new executable which runs a short test of sending and receiving
data through the HTTP/1 handler using multiple eventloops and showing
three options.
1) Vanilla
2) With in memory PCAP never written to disk
3) With a disk based PCAP.
Motivation:
Previously, when using the response compressor, doing a flush() right before finishing the
response data would cause the final compression chunk to be omitted. Some strict decompressors
(such as gzip or the zlib functionality exported in nodejs) would refuse to decompress the
incomplete response.
With this change, the generated compressed response is properly finalized.
Modifications:
In HTTPResponseCompressor.swift, a channel write is now also generated if no body data is
added, but a flush is required.
Result:
The response is now correct for this edge case, enabling gzip, nodejs, et al., to
decompress it without errors.
Motivation:
Capturing all packets is expensive. Recording to a ring buffer and
then outputting on a triggering event allows this cost to be reduced.
Modifications:
Add a new handler - NIOPCAPRingCaptureHandler.
This derives from the existing NIOWritePCAPHandler and generates PCAP recordings.
A ring buffer contained in this handler stores the captured packets until RecordPreviousPackets
is received as a user message at which point they are flushed to the sink.
Result:
There is a new handler capable of outputting packet captured data only in the build up to
a known event.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: George Barnett <gbrntt@gmail.com>
Motivation:
Previously, WritePCAPHandler would crash if more than 4GiB of data were
either received or sent through the same instance of the
WritePCAPHandler because of a UInt32 overflow representing the TCP
sequence/ACK numbers.
Modifications:
Make TCP sequence/ACK numbers wrap around correctly.
Result:
- now you can send/receive up to 16 EiB of data :P.
- fixes rdar://61887658
Motivation:
NIOHTTPRequestDecompressor and HTTPResponseDecompressor are both affected by an issue where the decompression limits defined by their DecompressionLimit property wasn't correctly checked when is was set with DecompressionLimit.size(...), allowing denial of service attacks.
Modifications:
- Update DecompressionLimit.size(...) to correctly check the size of the decompressed data.
- Update test cases to avoid future regressions regarding the size checks.
Result:
Prevents DoS attacks though maliciously crafted compressed data.
* Added NIOHTTPRequestCompressor to compress requests
Also moved common code from request and response compressor into separate NIOHTTPCompression enum.
* Updates after comments from @weissi
Also reinstated public enum HTTPResponseCompressor.CompressionError
* algorithms are now let not var
* Catch situation where head is flushed before anything else comes through
Content-encoding was not being set
Added additional tests for header values
* Added documentation around 5 bytes added to buffer size and add them
* Renaming NIOHTTPCompressionSetting to NIOCompression
Also
NIOHTTPCompressionSetting.CompressionAlgorithm is NIOCompression.Algorithm
NIOHTTPCompressionSetting.CompressionError is NIOCompression.Error
Algorithm now conforms to Equatable
* Forgot to run generate_linux_tests
* Fix typos
### Motivation:
There will be times when a client wishes to send larger requests with gzipped bodies to save on network traffic. This PR adds a `NIOHTTPRequestDecompressor` which can be added to the server's channel pipeline so those requests are automatically inflated.
### Modifications:
- Added a `CNIOExtrasZlib_voidPtr_to_BytefPtr` C method.
- Added a `NIOHTTPRequestDecompressor` type.
- Added a `HTTPResponseDecompressorTest` test case.
### Result:
Now you don't have to manually check the `Content-Encoding` header and decompress the body on each incoming request.
* extract common classes for server request decompressor
* review fix: make fields private and make state part of the handler
* review fixes
* review fix: reserve capacity before inflating
### Motivation:
Many HTTP servers can send compressed responses to clients and it would be a great feature for `AsyncHTTPClient` to support it. But since we want to minizime usage of unsafe APIs in SSWG projects, I propose to consolidate interfacing with zlib to `nio-extras` since it already supports `zlib` compression.
### Modifications:
Added `HTTPResponseDecompressor` and accompanying tests
### Result:
Users (primarily `AsyncHTTPClient`) can now provide automatic response decompression support.
Motivation:
ServerQuiescingHelper used to swallow close errors and it shoulnd't do
that.
Modifications:
Don't swallow close errors.
Result:
More correctness.
Motivation:
Use B2MDVerifier for the B2MDs in NIOExtras. Already found one bug,
separetely fixed in #51.
Modifications:
Write a basic validation test for all B2MDs.
Result:
Better test coverage.
Motivation:
JSPN-RPC uses various framing methods, one is Content-Length based
framing. NIOExtras should provide encoder/decoders for this.
Modifications:
Add such codecs.
Result:
NIOExtras more useful
Motivation:
LineBasedFrameDecoder previously would only correctly decode \r\n as a
line-ending iff \r\n were not split apart.
Modifications:
Handle \r\n arriving apart.
Result:
more correct line splitting
Motivation:
Especially with TLS but also without, in real production environments it
can be handy to be able to write pcap files from NIO directly.
Modifications:
add a ChannelHandler that can write a PCAP trace from what's going on in
the ChannelPipeline.
Result:
easier debugging in production
Motivation:
LineBasedFrameDecoder discarded everything after EOF and delivered it in
the left-over bytes error. For the real world however that doesn't make
much sense, you'd want all previously received lines and only receive
the partial lines as left-overs.
Modifications:
deliver lines until there are only partial lines left, even in case of
EOF.
Result:
LineBasedFrameDecoder more useful
Motivation:
HTTPResponseCompressor is trivially removable, so mark it.
Modifications:
make HTTPResponseCompressor implement RemovableChannelHandler
Result:
HTTPResponseCompressor can be removed
Motivation:
Currently, we crash if there's any left over bytes available because
LineBasedFrameDecoder modifies cumulationBuffer in a way that is
illegal.
Modifications:
stop modifying cumulationBuffer in an illegal way
Result:
fewer crashes
Motivation:
Frequently, people want to terminate their pipeline with a handler that
takes in requests & a promise and on receipt of the response just
fulfill that promise.
Modifications:
- add `RequestResponseHandler`
- remove outdated of contents from README.md
Result:
more useful handlers
Motivation:
Users may want to log all of inbound and/or outbound events
Modifications:
Add DebugInboundEventsHandler and DebugOutboundEventsHandler
Result:
Users can plug additional handlers into their pipeline for default printing of events or getting a hook with relevant information for their own logging mechanism.
* Adds a LengthFieldPrepender class to prepend the length onto a message.
This class is a type of byte to message encoder.
Motivation:
To encode a prepended length field on data so that messages of arbitrary size can be sent.
Can work as a pair with the ‘LengthFieldBasedFrameDecoder’.
Modifications:
Added ‘LengthFieldPrepender’
Added unit tests for ‘LengthFieldPrepender’ in ‘LengthFieldPrependerTest’
Updated the linux text files by running the script.
Result:
The length can now be easily prepended to any message.
Motivation:
ByteToMessageDecoder is extremely brittle, for example a reentrant call
into decodeLast will present the user with bytes that were previously
seen...
Modification:
Discard bytes in decodeLast
Result:
LengthFieldBasedFrameDecoder will work if close called from channelRead.
* Adds a basic LengthFieldBasedFrameDecoder
Motivation:
Adding a popular type of decoder that is useful in real-world situations, particularly when dealing with protocol buffers.
Modifications:
Added the decoder class, tests and linux test files.
Result:
The project now includes a basic length field based decoder which can be built upon.
Further header specification may be required but this version suits basic usage.
* Renames the class files for the frame decoders to match the class names.
Motivation:
Neatening of the project by ensuring file names match the contained class names.
Modifications:
5 file renames, 4 of which are test files.
Result:
Ruby hooks script ran with no changes.
No breaking changes.
The project is a little neater.
* Adds a line-based frame decoder that can split received buffers on line endings.
Motivation:
As per https://github.com/apple/swift-nio/issues/473
Modifications:
Added a new decoder (LineBasedFrameDecoder) that splits incoming buffers on line end characters.
Result:
Received buffers will be split on line end character(s) ('\n' or '\r\n'), with these characters
stripped in the resulting buffers.
Provide a decoder for frames with a fixed length.
Motivation:
This was motivated by the issue https://github.com/apple/swift-nio/issues/474 as a good first issue.
Modifications:
Implemented a FixedLengthFrameDecoder as well as tests.
Result:
Users can decode fixed length frames.
Motivation:
In a few cases quiescing a server application is useful but it's harder
than necessary with core-NIO. Therefore this adds a helper & a
demonstration.
Modifications:
- add `QuiescingHelper` which helps users to quiesce a channel by
collecting all accepted channels and when needed sends them the
quiescing user event. When all collected channels have closed the
user will be notified and can just shut down the ELG.
- added a demo implementation with a simple HTTP server that quiesces
when receiving a signal
Result:
Make it quite easy to quiesce a server and show users how to do it.
