MCorange/wings
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Modify backup process to utilize contexts and exponential backoffs

Browse Source 
If a request to upload a file part to S3 fails for any 5xx reason it will begin using an exponential backoff to keep re-trying the upload until we've reached a minute of trying to access the endpoint.

This should resolve temporary resolution issues with URLs and certain S3 compatiable systems such as B2 that sometimes return a 5xx error and just need a retry to be successful.

Also supports using the server context to ensure backups are terminated when a server is deleted, and removes the http call without a timeout, replacing it with a 2 hour timeout to account for connections as slow as 10Mbps on a huge file upload.
This commit is contained in:
Dane Everitt
2021-05-02 12:28:36 -07:00
parent da74ac8291
commit ddfd6d9cce
6 changed files with 239 additions and 169 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
server/backup.go
View File

@@ -66,7 +66,7 @@ func (s *Server) Backup(b backup.BackupInterface) error {
}
}
ad, err := b.Generate(s.Filesystem().Path(), ignored)
ad, err := b.Generate(s.Context(), s.Filesystem().Path(), ignored)
if err != nil {
if err := s.notifyPanelOfBackup(b.Identifier(), &backup.ArchiveDetails{}, false); err != nil {
s.Log().WithFields(log.Fields{
@@ -150,7 +150,7 @@ func (s *Server) RestoreBackup(b backup.BackupInterface, reader io.ReadCloser) (
// Attempt to restore the backup to the server by running through each entry
// in the file one at a time and writing them to the disk.
s.Log().Debug("starting file writing process for backup restoration")
err = b.Restore(reader, func(file string, r io.Reader) error {
err = b.Restore(s.Context(), reader, func(file string, r io.Reader) error {
s.Events().Publish(DaemonMessageEvent, "(restoring): "+file)
return s.Filesystem().Writefile(file, r)
})