Dane Everitt
ddfd6d9cce
Modify backup process to utilize contexts and exponential backoffs
...
If a request to upload a file part to S3 fails for any 5xx reason it will begin using an exponential backoff to keep re-trying the upload until we've reached a minute of trying to access the endpoint.
This should resolve temporary resolution issues with URLs and certain S3 compatiable systems such as B2 that sometimes return a 5xx error and just need a retry to be successful.
Also supports using the server context to ensure backups are terminated when a server is deleted, and removes the http call without a timeout, replacing it with a 2 hour timeout to account for connections as slow as 10Mbps on a huge file upload.
2021-05-02 12:28:36 -07:00
Dane Everitt
98c68142cd
Remove all of the remaining API logic and port it all to the remote.Client type
2021-02-01 21:28:46 -08:00
Dane Everitt
4bd18f7dd8
Send restoration status information to the console
2021-01-30 18:49:07 -08:00
Dane Everitt
adc0732af3
Better error handling and logging for restorations
2021-01-30 18:43:35 -08:00
Dane Everitt
3f84ee694b
Get backups restoring kinda
2021-01-20 20:03:14 -08:00
Dane Everitt
63dac51692
Include a better stack here
2021-01-18 21:22:37 -08:00
Dane Everitt
6ef0bd7496
Get general concept for backup resotration using a unified interface implemented
2021-01-18 21:20:58 -08:00
Jakob Schrettenbrunner
648072436f
organize imports with gopls format
2021-01-10 01:22:39 +00:00
Matthew Penner
3a7c4822f8
Improve logged stacktraces
2020-12-27 13:55:58 -07:00
Dane Everitt
31d4c1d34f
Close the file when done
2020-12-25 14:01:25 -08:00
Dane Everitt
0cdfdc725c
Cleanup ignored file reader
2020-12-25 14:00:01 -08:00
Matthew Penner
a450abc080
Don't read directories either, add completion log for transfers
2020-12-25 14:46:33 -07:00
Matthew Penner
e05c601325
Lets not read an entire file into memory
2020-12-25 14:40:53 -07:00
Matthew Penner
901ab1157d
Rework archiving logic to be more consistent and less impactful on disk IO ( #79 )
...
Co-authored-by: Dane Everitt <dane@daneeveritt.com>
2020-12-25 11:52:57 -08:00
Dane Everitt
59a111de39
Invite our lord and savior emperror back to the table
2020-12-15 21:56:53 -08:00
Matthew Penner
ee08829a28
s3 backups: handle CompleteMultipartUpload and AbortMultipartUpload on the panel
2020-12-06 13:56:17 -07:00
Matthew Penner
de51fd1c51
Error handling improvements ( #71 )
...
* Remove `emperror.dev/errors`, remove all `errors#Wrap` and `errors#WithStack` calls
* Improve logging in `server/backup.go`
2020-11-28 16:57:10 -07:00
Dane Everitt
ef999a039c
Perhaps better error stacks for backups/archives; ref #2418
2020-11-08 14:07:26 -08:00
Dane Everitt
be9d1a3986
Replace error handling package with emperror; add better reporting for errors escaping server root
2020-11-08 13:52:20 -08:00
Dane Everitt
334b3e8d10
Refactor HTTP endpoints to be less complicated and follow better standards
2020-10-31 10:04:20 -07:00
Dane Everitt
0f7bb1a371
Refactor filesystem to not be dependent on a server struct
2020-09-27 12:24:08 -07:00
Dane Everitt
a7ca6b2e34
Correctly report backup API errors; ref pterodactyl/panel#2271
2020-08-27 19:35:57 -07:00
Dane Everitt
3489088703
Update output from socket to use checksum
2020-08-23 18:06:17 -07:00
Dane Everitt
138080d5e2
return success state of the backup from wings
2020-08-20 21:28:06 -07:00
Dane Everitt
7d4a8d7f7e
Update everything expect transfers & sftp to not use zap
2020-06-13 10:40:26 -07:00
Dane Everitt
0bd28a4480
Cleanup S3 support; send actual backup details in response
2020-05-09 19:24:30 -07:00
Dane Everitt
b2797ed292
Abstract out shared backup functionality
2020-05-02 15:02:02 -07:00
Dane Everitt
91d12ab9a7
More abstract support for backups & misc code cleanup in that area
2020-04-26 16:43:18 -07:00
Dane Everitt
6395b8b56c
Support for .pteroignore files when generating backups
2020-04-19 19:52:19 -07:00
Dane Everitt
df6d98bbda
Add support for ignoring directories/files; fix compression of archives
2020-04-18 23:26:23 -07:00
Dane Everitt
3bca54655b
Adjust callback to support continuing or stopping at current depth
2020-04-18 18:51:19 -07:00
Dane Everitt
4ff7bd2777
Abstract out more of the backup functionality
2020-04-17 13:46:36 -07:00
Dane Everitt
11035b561a
Break out the backup functions of the daemon in prep for S3 support
2020-04-13 22:01:07 -07:00
Dane Everitt
33875105b6
Fix topic handling for websocket with namespace
2020-04-06 21:22:43 -07:00
Dane Everitt
222091b68c
Fire an event to the websocket when a backup is completed
2020-04-06 21:03:50 -07:00
Dane Everitt
03311ecf03
Move over backup and create transfer logic
2020-04-05 19:07:16 -07:00
Dane Everitt
4ad57af990
Support one-time downloads of server backups
2020-04-04 19:55:23 -07:00
Dane Everitt
4ce2b73490
Add first implementation of local backups for a server
2020-04-04 16:07:25 -07:00
