MCorange/wings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Modify backup process to utilize contexts and exponential backoffs

Browse Source 
If a request to upload a file part to S3 fails for any 5xx reason it will begin using an exponential backoff to keep re-trying the upload until we've reached a minute of trying to access the endpoint.

This should resolve temporary resolution issues with URLs and certain S3 compatiable systems such as B2 that sometimes return a 5xx error and just need a retry to be successful.

Also supports using the server context to ensure backups are terminated when a server is deleted, and removes the http call without a timeout, replacing it with a 2 hour timeout to account for connections as slow as 10Mbps on a huge file upload.
This commit is contained in:
Dane Everitt
2021-05-02 12:28:36 -07:00
parent da74ac8291
commit ddfd6d9cce
6 changed files with 239 additions and 169 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
go.mod
View File

@@ -14,6 +14,7 @@ require (
github.com/asaskevich/govalidator v0.0.0-20200907205600-7a23bdc65eef
github.com/beevik/etree v1.1.0
github.com/buger/jsonparser v1.1.0
github.com/cenkalti/backoff/v4 v4.1.0
github.com/cobaugh/osrelease v0.0.0-20181218015638-a93a0a55a249
github.com/containerd/containerd v1.4.3 // indirect
github.com/containerd/fifo v0.0.0-20201026212402-0724c46b320c // indirect