Support HTTP resumable upload.
### Motivation:
Supporting HTTP resumable upload protocol defined in
https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-resumable-upload-05
* Interop version 3: iOS 17.0, macOS 14.0
* Interop version 5: iOS 18.0, macOS 15.0
* Interop version 6: iOS 18.1, macOS 15.1
### Modifications:
2 new public classes, `HTTPResumableUploadHandler` and
`HTTPResumableUploadContext`, and a few other supporting objects to
manage resumable uploads and translate them into regular uploads.
---------
Co-authored-by: Jonathan Flat <jflat@apple.com>
Co-authored-by: Cory Benfield <lukasa@apple.com>
### Motivation:
To migrate to GitHub actions and centralised infrastructure.
### Modifications:
Changes of note:
* Adopt swift-format using rules from SwiftNIO
* Remove scripts and docker files which are no longer needed
### Result:
Feature parity with old CI.
### Motivation:
`LineBasedFrameDecoder.decodeLast` throws an error if there is more than
one frame’s worth of bytes remaining in the buffer.
This is in violation of the `NIOSingleStepByteToMessageDecoder`protocol
requirement that this method be called in a loop until all bytes are
decoded.
### Modifications:
* The method now only throws if the decode operation could not return a
frame and there are bytes left in the buffer (previously only the latter
criterion applied).
* Method documentation is updated.
* Test added.
### Result:
`LineBasedFrameDecoder.decodeLast` can cope with more edge cases.
Co-authored-by: Rick Newton-Rogers <rnro@apple.com>
* Added support for conditional response compression
* Updated expectations to be fulfilled via a defer
* Updated compression response predicate to return an intent enum rather than a boolean
Motivation:
`NIOHTTPDecompression` erroneously accumulates decompressed data sizes across multiple requests in a single channel, leading to unwarranted `DecompressionError.limit` errors. This affects applications using persistent connections, as the decompression limits are improperly enforced. This change aims to address and rectify this issue.
Modifications:
- Add initialization of `inflated` within `Decompressor.initializeDecoder`.
- Introduced new tests to validate decompression functionality across multiple requests on the same channel
Result:
This fix ensures each request's decompression size is independently considered, eliminating incorrect limit errors, and enhancing reliability for applications using HTTP compression with persistent connections.
Motivation
Nightly CI builds require annotations on retroactive conformances.
We have a few used only in tests, which are totally safe.
Modifications
Add retroactive conformance marker.
Result
Nightly CI works again
Motivation:
The NIOPCAPRingBuffer can limit the number of fragment or the total
number of bytes in its buffer or both. When configuring the buffer to
limit only the maximum number of bytes it sets the maximum number of
fragments allowed to `.max`. On `init` the buffer has enough capacity
reserved to store tha maximum number of fragments.
This would be a large and potentially totally unnecessary allocation.
That is, if it didn't crash at runtime. It crashes at runtime as
`CircularBuffer` converts the requested capacity to a `UInt32` which
traps if you pass it an `Int.max`.
Modifications:
- Don't reserve capacity on init
- Adjust the test which tests the byte limit to not set a capacity as
well
Result:
- `NIOPCAPRingBuffer(maximumBytes:)` doesn't crash
* Add HTTP types adapter for SwiftNIO
* swiftformat
* Guard on Swift 5.8
* Review comments
* Update swift-http-types to 0.1.1
* Update swift-http-types to 1.0.0
* Review feedback
* Review feedback
* Bump minimum Swift version to 5.7.1
* Allow Host in any order
Motivation
Per SwiftNIO's formal version policy, we are ready to drop support for
Swift 5.5.
Modifications
This patch removes the support for 5.5 and all supporting
infrastructure. This includes the test generation functionality, which
is no longer required, as well as the files generated by that
functionality. It updates the dockerfile for 5.8, and it removes all
conditional compilation checks that are now definitionally true.
Result
A nice, clean, 5.6+ codebase
Motivation
syncClose will block whatever thread it's on indefinitely. That makes it
unsafe to call in async contexts.
Modifications
Add a new close() method that's async.
Make the existing method unavailable from async.
Add some tests.
Results
Easier to close these from async contexts
# Motivation
Currently the `QuiescingHelper` is crashing on a precondition if you call shutdown when it already was shutdown. However, that can totally happen and we should support it.
# Modification
Refactor the `QuiescingHelper` to exhaustively switch over its state in every method. Furthermore, I added a few more test cases to test realistic scenarios.
# Result
We are now reliable checking our state and making sure to allow most transitions.
* ServerQuiescingHelper no longer leaking promises
Motivation:
ServerQuiescingHelper leaked promises when promise left scope and not succeeded
Modifications:
Failing promise within a deinit
* Minor fixes
* generating linux tests
* mini update
Motivation:
Moving the HTTP1ProxyConnectHandler into swift-nio-extras will make the
code which is generally useful when dealing with HTTP1 proxies available
more easily to a wider audience.
Modifications:
The code and tests are copied over from 0b5bec741b/Sources/AsyncHTTPClient/ConnectionPool/ChannelHandler/HTTP1ProxyConnectHandler.swift.
Result:
HTTP1ProxyConnectHandler will be surfaced via the NIOExtras library
Motivation:
SwiftNIO periodically drops support for older Swift versions. Now that
5.7 has been released, 5.4 will be dropped.
Modifications:
- Remove 5.4 specific Package.swift and docker-compose
- Update the 5.7 docker-compose to use the released 5.7 and move from
focal (2004) to jammy (2204)
- Update docs
Results:
Minimum Swift version is 5.5
Motivation
Currently we don't confirm that the decompression has completed
successfully. This means that we can incorrectly spin forever attempting
to decompress past the end of a message, and that we can fail to notice
that a message is truncated. Neither of these is good.
Modifications
Propagate the message zlib gives us as to whether or not decompression
is done, and keep track of it.
Add some tests written by @vojtarylko to validate the behaviour.
Result
Correctly police the bounds of the messages.
Resolves#175 and #176.
Motivation:
With NIO 2.32.0 we broke the core NIO module up into modules that split
apart the POSIX layer and the core abstractions. As a result, this
package no longer needs to express a hard dependency on the POSIX layer.
Modifications:
- Rewrote imports of NIO to NIOCore.
- Added NIOEmbedded and NIOPosix imports where necessary in tests.
- Extended soundness script to detect NIO imports.
- Note that the main modules still depend on NIO, which is necessary
for backwards-compatibility reasons. This dependency is unused.
Result:
No need to use NIOPosix.
* SOCKS handshake handler implementation
* Soundness
* Remove placeholder text
* Sad path tests
* Soundness
* Docs
* Fix workflow tests
* Make handler removable
* Protect methods
* Prevent test crashes
* Cleanup public types
* Add test that writing after auth fails
* Add force handler removal tests
* Remove client and server state from public api
* Explicitly handle states
* Remove promises
* Fix test
* Add data to authentication complete
* Refactor to add authentication complete flag
Implement a SOCKSv5 client according to RFC 1928. Server implementation will be added in another PR to keep sizes down.
https://datatracker.ietf.org/doc/html/rfc1928
A few meaningful changes:
Add all relevant types used across SOCKS clients and servers
Add a state machine used to manage a connection from a clients side
Add a channel handler that should be added at the very start of a channel pipeline
* fix crash in LengthFieldBasedFrameDecoder for malicious length values
Motivation:
LengthFieldBasedFrameDecoder will cause a fatal error if the length value does not fit into an `Int`.
This can happen if `lengthFieldLength` is set to `.eight` and we are on a 64 bit platform or if `lengthFieldLength` is set to `.four` and we are on a 32-bit platform.
If we then receive a length field value which is greater than `Int.max` the conversion from `UInt` to `Int` will cause a fatal error.
This could be abused to crash a server by only sending 4 or 8 bytes.
Modifications:
safely convert UInt64 & UInt32 to Int and throw an error if they can't be represented as an Int
Result:
- LengthFieldBasedFrameDecoder with lengthFieldLength set to `.eight` can no longer crash the server on a 64-bit platform
- LengthFieldBasedFrameDecoder with lengthFieldLength set to `.four` can no longer crash the server on a 32-bit platform
* use early exit instead of XCTSkipIf
* add support for `.eight` on 32-bit platforms
* limit frame length to `Int32.max`
* change test names
* throw correct error
* fix compilation for Swift 5.0 and add NIO prefix to error enum
* add test for maximum allowed length and one above the maximum allowed length
Signed-off-by: David Nadoba <dnadoba@gmail.com>
* run XCTest script
Signed-off-by: David Nadoba <dnadoba@gmail.com>
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Motivation:
The RSocket protocol uses a 24 bit length field
Modifications:
- add two new methods readInteger and writeInteger on ByteBuffer that support reading and writing integers of any size.
- add a new case (.three) to ByteLength
Result:
LengthFieldBasedFrameDecoder & LengthFieldPrepender do now support a 24 bit length field
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Motivation:
It's possible for channels to be closed without an error; and the
`RequestResponseHandler` should tolerate that by failing any promises
for which it does not have a response for.
Modifications:
- Add `ClosedBeforeReceivingResponseError`
- Fail outstanding promises with `ClosedBeforeReceivingResponseError` in
`RequestResponseHandler.channelInactive`
- Add a test.
Result:
Outstanding request promises are failed when the channel becomes inactive.
Motivation:
Previously, when using the response compressor, doing a flush() right before finishing the
response data would cause the final compression chunk to be omitted. Some strict decompressors
(such as gzip or the zlib functionality exported in nodejs) would refuse to decompress the
incomplete response.
With this change, the generated compressed response is properly finalized.
Modifications:
In HTTPResponseCompressor.swift, a channel write is now also generated if no body data is
added, but a flush is required.
Result:
The response is now correct for this edge case, enabling gzip, nodejs, et al., to
decompress it without errors.
Motivation:
Capturing all packets is expensive. Recording to a ring buffer and
then outputting on a triggering event allows this cost to be reduced.
Modifications:
Add a new handler - NIOPCAPRingCaptureHandler.
This derives from the existing NIOWritePCAPHandler and generates PCAP recordings.
A ring buffer contained in this handler stores the captured packets until RecordPreviousPackets
is received as a user message at which point they are flushed to the sink.
Result:
There is a new handler capable of outputting packet captured data only in the build up to
a known event.
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: George Barnett <gbrntt@gmail.com>
Motivation:
Previously, WritePCAPHandler would crash if more than 4GiB of data were
either received or sent through the same instance of the
WritePCAPHandler because of a UInt32 overflow representing the TCP
sequence/ACK numbers.
Modifications:
Make TCP sequence/ACK numbers wrap around correctly.
Result:
- now you can send/receive up to 16 EiB of data :P.
- fixes rdar://61887658
Motivation:
NIOHTTPRequestDecompressor and HTTPResponseDecompressor are both affected by an issue where the decompression limits defined by their DecompressionLimit property wasn't correctly checked when is was set with DecompressionLimit.size(...), allowing denial of service attacks.
Modifications:
- Update DecompressionLimit.size(...) to correctly check the size of the decompressed data.
- Update test cases to avoid future regressions regarding the size checks.
Result:
Prevents DoS attacks though maliciously crafted compressed data.
* Added NIOHTTPRequestCompressor to compress requests
Also moved common code from request and response compressor into separate NIOHTTPCompression enum.
* Updates after comments from @weissi
Also reinstated public enum HTTPResponseCompressor.CompressionError
* algorithms are now let not var
* Catch situation where head is flushed before anything else comes through
Content-encoding was not being set
Added additional tests for header values
* Added documentation around 5 bytes added to buffer size and add them
* Renaming NIOHTTPCompressionSetting to NIOCompression
Also
NIOHTTPCompressionSetting.CompressionAlgorithm is NIOCompression.Algorithm
NIOHTTPCompressionSetting.CompressionError is NIOCompression.Error
Algorithm now conforms to Equatable
* Forgot to run generate_linux_tests
* Fix typos
