Daniel Barton
d1fd0465e4
server(filesystem): fix file names when extracting archives ( #165 )
2023-10-17 20:47:36 -06:00
Matthew Penner
e9b8b11fec
Ensure files are closed after they are done being used
2023-01-17 18:34:08 -07:00
Matthew Penner
57e7eb714c
Streaming Transfers ( #153 )
2022-11-14 18:25:01 -07:00
Matthew Penner
eb4df39d14
server(filesystem): fix inaccurate archive progress ( #145 )
2022-11-06 13:38:30 -07:00
Matthew Penner
7245791214
backups(s3): send uploaded parts in complete request
2022-09-26 11:14:57 -06:00
Matthew Penner
5cd43dd4c9
archive: keep timestamps when extracting
2021-09-01 09:54:41 -06:00
Matthew Penner
3c54c1f840
break everything
...
- upgrade dependencies
- run gofmt and goimports to organize code
- fix typos
- other small tweaks
2021-08-02 15:07:00 -06:00
Matthew Penner
31ff3f8b56
server(fs): keep file mode when extracting archive
2021-07-15 15:37:38 -06:00
Dane Everitt
ddfd6d9cce
Modify backup process to utilize contexts and exponential backoffs
...
If a request to upload a file part to S3 fails for any 5xx reason it will begin using an exponential backoff to keep re-trying the upload until we've reached a minute of trying to access the endpoint.
This should resolve temporary resolution issues with URLs and certain S3 compatiable systems such as B2 that sometimes return a 5xx error and just need a retry to be successful.
Also supports using the server context to ensure backups are terminated when a server is deleted, and removes the http call without a timeout, replacing it with a 2 hour timeout to account for connections as slow as 10Mbps on a huge file upload.
2021-05-02 12:28:36 -07:00
Dane Everitt
daaef5044e
Correctly determine name for archive files when decompressing; closes pterodactyl/panel#3296
2021-04-25 15:36:00 -07:00
Dane Everitt
0676a82a21
Add better error handling for filesystem
2021-04-17 13:29:18 -07:00
Matthew Penner
0e3778ac47
transfers: use backup archiver
2021-03-07 11:04:15 -07:00
Dane Everitt
98c68142cd
Remove all of the remaining API logic and port it all to the remote.Client type
2021-02-01 21:28:46 -08:00
Dane Everitt
6ef0bd7496
Get general concept for backup resotration using a unified interface implemented
2021-01-18 21:20:58 -08:00
Dane Everitt
66b6f40b61
Fix import cycle issue
2021-01-17 21:05:51 -08:00
Dane Everitt
7dd0acebc0
Initial untested pass at restoring from local backups
2021-01-16 18:06:22 -08:00
Dane Everitt
902f9f5944
Include more (and customizable) context with backup logs
2020-12-27 16:16:40 -08:00
Matthew Penner
de4d2f4724
Make logs follow a more consistent format
2020-12-27 13:01:34 -07:00
Matthew Penner
640e30de8a
Ignore symlinks with missing target, add better backup logging, update copyright year to 2021
2020-12-27 12:54:18 -07:00
Matthew Penner
901ab1157d
Rework archiving logic to be more consistent and less impactful on disk IO ( #79 )
...
Co-authored-by: Dane Everitt <dane@daneeveritt.com>
2020-12-25 11:52:57 -08:00
Matthew Penner
de51fd1c51
Error handling improvements ( #71 )
...
* Remove `emperror.dev/errors`, remove all `errors#Wrap` and `errors#WithStack` calls
* Improve logging in `server/backup.go`
2020-11-28 16:57:10 -07:00
Dane Everitt
be9d1a3986
Replace error handling package with emperror; add better reporting for errors escaping server root
2020-11-08 13:52:20 -08:00
Dane Everitt
de30e2fcc9
Dont attempt to get size within archive process, will return empty; ref pterodactyl/panel#2420
...
The stat call is operating against an unflushed file if called in the archive function, so you'll just get the emtpy archive size, rather than the final size.
Plus, we only used the file stat in one place, so slight efficiency win?
2020-09-27 11:16:38 -07:00
Dane Everitt
5f1d9ff151
Don't throw a 500 error if the backup is missing; closes pterodactyl/panel#2280
2020-08-31 20:24:07 -07:00
Dane Everitt
0e474c8b24
Better error captures
2020-08-27 19:57:22 -07:00
Dane Everitt
c1e591c99b
Support generating a compressed archive for a server via the API
2020-07-11 13:13:49 -07:00
Dane Everitt
0bd28a4480
Cleanup S3 support; send actual backup details in response
2020-05-09 19:24:30 -07:00
Dane Everitt
b2797ed292
Abstract out shared backup functionality
2020-05-02 15:02:02 -07:00
Dane Everitt
507d0100cf
Hilariously rough code to get a backup pushed into S3
2020-04-26 17:20:26 -07:00
Dane Everitt
91d12ab9a7
More abstract support for backups & misc code cleanup in that area
2020-04-26 16:43:18 -07:00
Dane Everitt
df6d98bbda
Add support for ignoring directories/files; fix compression of archives
2020-04-18 23:26:23 -07:00
Dane Everitt
4ff7bd2777
Abstract out more of the backup functionality
2020-04-17 13:46:36 -07:00
Dane Everitt
11035b561a
Break out the backup functions of the daemon in prep for S3 support
2020-04-13 22:01:07 -07:00
