MCorange/wings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,218 Commits 42 Branches 73 Tags 43 MiB
Commit Graph

511 Commits

Author SHA1 Message Date
Dane Everitt
ddfd6d9cce Modify backup process to utilize contexts and exponential backoffs 
If a request to upload a file part to S3 fails for any 5xx reason it will begin using an exponential backoff to keep re-trying the upload until we've reached a minute of trying to access the endpoint.

This should resolve temporary resolution issues with URLs and certain S3 compatiable systems such as B2 that sometimes return a 5xx error and just need a retry to be successful.

Also supports using the server context to ensure backups are terminated when a server is deleted, and removes the http call without a timeout, replacing it with a 2 hour timeout to account for connections as slow as 10Mbps on a huge file upload.
 2021-05-02 12:28:36 -07:00
Julien Tant
35b2c420ec add decompress tests 2021-04-25 16:44:54 -07:00
Dane Everitt
daaef5044e Correctly determine name for archive files when decompressing; closes pterodactyl/panel#3296 2021-04-25 15:36:00 -07:00
Dane Everitt
0676a82a21 Add better error handling for filesystem 2021-04-17 13:29:18 -07:00
Dane Everitt
4b244e96fb Fix .rar file decompression; closes pterodactyl/panel#3267 2021-04-17 13:13:37 -07:00
Dane Everitt
f57c24002e More API response fixing 2021-04-04 10:20:27 -07:00
Dane Everitt
2e0496c1f9 Add note about handling of UTF-8 sequences in properties files. 2021-04-03 14:02:37 -07:00
Dane Everitt
76b7967fef
Merge pull request #88 from Antony1060/develop 
Added app name
 2021-04-03 11:13:29 -07:00
Dane Everitt
1b1eaa3171 Avoid expensive copies of the config for every line output 2021-04-03 11:11:36 -07:00
Dane Everitt
b448310a33 Correctly return servers installed on wings and their resource usage 2021-04-03 11:08:26 -07:00
antony1060
0c17e240f4
Added app name 2021-03-24 10:26:03 +01:00
Matthew Penner
471886dd34 internally mark if a server is restoring to restrict actions 2021-03-12 16:19:35 -07:00
Matthew Penner
0e3778ac47 transfers: use backup archiver 2021-03-07 11:04:15 -07:00
Dane Everitt
fb2dc39a47 Remove some forgotten debug 2021-03-03 21:00:58 -08:00
Dane Everitt
0919fb2da6 Improve error handling and reporting for server installation & process boot 2021-03-03 20:56:18 -08:00
Dane Everitt
2eb721bbe7 Merge branch 'dane/api-cleanup' into develop 2021-02-23 21:25:10 -08:00
Matthew Penner
5c56ddf5d6 fs: only mkdir and chown if not exists 2021-02-10 13:58:47 -07:00
Matthew Penner
683c766d0f fs: fix potential panic when chowning a file 2021-02-05 13:30:02 -07:00
Dane Everitt
1da415c177
Revert "Don't tempt people with this public interface" 
This reverts commit e3b0b91912.
 2021-02-01 21:43:10 -08:00
Dane Everitt
e3b0b91912
Don't tempt people with this public interface 2021-02-01 21:33:03 -08:00
Dane Everitt
bfff094216
Get wings booting properly 2021-02-01 21:32:34 -08:00
Dane Everitt
98c68142cd
Remove all of the remaining API logic and port it all to the remote.Client type 2021-02-01 21:28:46 -08:00
Dane Everitt
62cbe5e135
Migrate SFTP endpoints 2021-02-01 20:59:17 -08:00
Dane Everitt
6775c17324
Nuke more API code and begin consolidation process 2021-02-01 20:50:23 -08:00
Dane Everitt
1393937904
Fix race condition and flawed logic mis-querying panel for servers; closes pterodactyl/panel#3059 2021-02-01 20:26:15 -08:00
Matthew Penner
de9b413bc2 Call cancel function for context to not leak resources 2021-02-01 20:09:24 -07:00
Dane Everitt
ba6cec9615
Merge branch 'dane/backup-restore' into develop 2021-01-30 18:51:35 -08:00
Dane Everitt
4bd18f7dd8
Send restoration status information to the console 2021-01-30 18:49:07 -08:00
Dane Everitt
adc0732af3
Better error handling and logging for restorations 2021-01-30 18:43:35 -08:00
Dane Everitt
df721f45f8
Reset the whole build block to avoid merge issues with 0 values 2021-01-27 21:41:13 -08:00
Dane Everitt
4411493006
Merge branch 'develop' into dane/backup-restore 2021-01-25 20:32:32 -08:00
Dane Everitt
31757a68a9
Fix race condition 2021-01-25 20:29:34 -08:00
Dane Everitt
f3a6ee7a45
re-refactor code 2021-01-25 20:28:24 -08:00
Dane Everitt
ab86fb703a
Merge branch 'develop' into schrej/refactor 2021-01-25 19:31:16 -08:00
Dane Everitt
981071cda8
Merge branch 'develop' into dane/backup-restore 2021-01-25 19:17:14 -08:00
Dane Everitt
93506994a5
Ensure the root directory for a server is always create when booting wings 2021-01-23 10:45:29 -08:00
Jakob Schrettenbrunner
82f70c2755 derp: rename panelapi to remote 2021-01-22 22:38:11 +00:00
Dane Everitt
5d070cbdc5
Handle edge case where a user triggers an install when the server has no data directory 2021-01-21 20:58:52 -08:00
Dane Everitt
3f84ee694b
Get backups restoring kinda 2021-01-20 20:03:14 -08:00
Dane Everitt
63dac51692
Include a better stack here 2021-01-18 21:22:37 -08:00
Dane Everitt
6ef0bd7496
Get general concept for backup resotration using a unified interface implemented 2021-01-18 21:20:58 -08:00
Dane Everitt
66b6f40b61
Fix import cycle issue 2021-01-17 21:05:51 -08:00
Dane Everitt
7dd0acebc0
Initial untested pass at restoring from local backups 2021-01-16 18:06:22 -08:00
Dane Everitt
2968ea3498
Modify stat to embed os.FileInfo differently and update file content reader 2021-01-16 12:03:55 -08:00
Dane Everitt
67ecbd667a
Minor improvements to logic around decompression 2021-01-16 11:48:30 -08:00
Dane Everitt
05c04c4350
Update remaining logic using viper 2021-01-14 20:19:28 -08:00
Dane Everitt
80faea3286
yoink viper back out of code, simplify some config logic 2021-01-14 20:11:01 -08:00
Dane Everitt
d45a159456
Fix tests 2021-01-12 20:07:00 -08:00
Dane Everitt
e9e70b6081
Better error handling; skip file when unarchiving 2021-01-10 17:01:41 -08:00
Dane Everitt
b10e4dd437
Better error handling for access to denylist files 2021-01-10 16:43:33 -08:00
Dane Everitt
2c1b211280
Add base idea for denying write access to certain files; ref pterodactyl/panel#569 2021-01-10 16:33:39 -08:00
Dane Everitt
b0fa9619de
debug 2021-01-10 15:09:33 -08:00
Dane Everitt
c228acaafc
Massive refactor of SFTP system now that it is deeply integrated with Wings 2021-01-10 14:25:39 -08:00
Jakob Schrettenbrunner
8192244fec replace servers.servers singleton with server.Manager 2021-01-10 02:37:02 +01:00
Jakob Schrettenbrunner
648072436f organize imports with gopls format 2021-01-10 01:22:39 +00:00
Matthew Penner
66c9be357c Potential fix for servers being marked as stopping after being marked as offline 2021-01-07 19:32:15 -07:00
Dane Everitt
d6e189df5e
Update commentary 2020-12-27 16:53:40 -08:00
Dane Everitt
68749616ad
Ignore symlink errors 2020-12-27 16:30:00 -08:00
Dane Everitt
7549eb13a0
Merge branch 'develop' of https://github.com/pterodactyl/wings into develop 2020-12-27 16:17:17 -08:00
Dane Everitt
902f9f5944
Include more (and customizable) context with backup logs 2020-12-27 16:16:40 -08:00
Matthew Penner
2cf24994d9 Uncomment important code 2020-12-27 13:56:45 -07:00
Matthew Penner
3a7c4822f8 Improve logged stacktraces 2020-12-27 13:55:58 -07:00
Matthew Penner
b26db99ee7 Fix part_id field in logs 2020-12-27 13:02:50 -07:00
Matthew Penner
de4d2f4724 Make logs follow a more consistent format 2020-12-27 13:01:34 -07:00
Matthew Penner
640e30de8a Ignore symlinks with missing target, add better backup logging, update copyright year to 2021 2020-12-27 12:54:18 -07:00
Matthew Penner
e75118e0f0 Maybe this will fix shit? 2020-12-27 12:21:26 -07:00
Matthew Penner
a4c8b8714b Bump all dependencies 2020-12-27 11:49:08 -07:00
Dane Everitt
22c53c365a
Fix installer to not explode on long lines; closes pterodactyl/panel#2773 2020-12-25 17:05:01 -08:00
Dane Everitt
59c30c2842
Fix use of atomics in codebase 2020-12-25 17:04:18 -08:00
Dane Everitt
31d4c1d34f
Close the file when done 2020-12-25 14:01:25 -08:00
Dane Everitt
0cdfdc725c
Cleanup ignored file reader 2020-12-25 14:00:01 -08:00
Dane Everitt
bb132243ed
Merge branch 'develop' of https://github.com/pterodactyl/wings into develop 2020-12-25 13:57:33 -08:00
Dane Everitt
d96115325a
Add file helper 2020-12-25 13:57:26 -08:00
Matthew Penner
a450abc080 Don't read directories either, add completion log for transfers 2020-12-25 14:46:33 -07:00
Matthew Penner
e05c601325 Lets not read an entire file into memory 2020-12-25 14:40:53 -07:00
Matthew Penner
5c78cb9ab3
Add transfer logging output (#77) 
Co-authored-by: Dane Everitt <dane@daneeveritt.com>
 2020-12-25 13:32:41 -08:00
Matthew Penner
901ab1157d
Rework archiving logic to be more consistent and less impactful on disk IO (#79) 
Co-authored-by: Dane Everitt <dane@daneeveritt.com>
 2020-12-25 11:52:57 -08:00
Dane Everitt
c0523df696
Add a server context that gets canceled when a server is deleted 2020-12-25 11:21:09 -08:00
Dane Everitt
f8282c56cb
Avoid race conditions from mismatched use of atomic & sync.Mutex 2020-12-20 13:13:10 -08:00
Dane Everitt
c718da20e3
Check disk space before trying a write from the downloader; don't make empty directories if we can't even write the file 2020-12-20 11:17:53 -08:00
Dane Everitt
9c53436470
Check disk before even trying to run the download 2020-12-20 11:08:01 -08:00
Dane Everitt
3495fb1c76
Less racey logic for resource usage 2020-12-16 22:03:35 -08:00
Dane Everitt
16118874cf
Don't obliterate the lock 2020-12-16 20:21:22 -08:00
Dane Everitt
59a111de39
Invite our lord and savior emperror back to the table 2020-12-15 21:56:53 -08:00
Dane Everitt
3a26a5d39d
Convert all filesystem error types into the same Error struct 2020-12-15 20:51:13 -08:00
Matthew Penner
8e29ffed50
Add configurable disk write speed limit for backups (#74) 
* Add configurable disk write speed limit for backups
 2020-12-08 09:13:48 -07:00
Dane Everitt
a18f60bd05
Merge branch 'develop' of https://github.com/pterodactyl/wings into develop 2020-12-06 15:35:04 -08:00
Dane Everitt
deea5babbc
Publish message when error occurs during crash handling 2020-12-06 15:34:58 -08:00
Matthew Penner
b52c3fb61e Cleanup backup/backup_s3.go 2020-12-06 15:25:11 -07:00
Matthew Penner
ee08829a28 s3 backups: handle CompleteMultipartUpload and AbortMultipartUpload on the panel 2020-12-06 13:56:17 -07:00
Matthew Penner
ba0a1a651e Only use the permission bits for ModeBits 2020-11-29 16:13:07 -07:00
Matthew Penner
068f41393d Return mode_bits when listing files in a directory 2020-11-29 14:20:24 -07:00
Matthew Penner
f62f714863 Add chmod endpoint to server files 2020-11-29 13:07:45 -07:00
Matthew Penner
de51fd1c51
Error handling improvements (#71) 
* Remove `emperror.dev/errors`, remove all `errors#Wrap` and `errors#WithStack` calls
* Improve logging in `server/backup.go`
 2020-11-28 16:57:10 -07:00
Matthew Penner
40c70673cd Use registry authentication when pulling install image, fixes https://github.com/pterodactyl/panel/issues/2589 2020-11-20 14:36:29 -07:00
Matthew Penner
73b221d022 Switch old Server#GetState() calls over to Server#Environment#State() 2020-11-20 14:35:29 -07:00
Matthew Penner
287b286940 Oh, that's why it was done like that 2020-11-18 18:44:25 -07:00
Matthew Penner
1d0e85cf55 Attempt to fix 'archive/tar: write too long' error when creating a backup 2020-11-18 16:30:34 -07:00
Dane Everitt
8081c83de4
Allow disabling disk checking with 0 value for check interval 2020-11-12 20:50:03 -08:00
Matthew Penner
2197d82957 Clean paths from AllowedMounts to ensure consistency 2020-11-09 16:58:01 -07:00