Compare commits

..

12 Commits
v1.7.5 ... v2

Author SHA1 Message Date
Matthew Penner
ee280f1deb Merge branch 'develop' into v2 2022-04-01 09:48:24 -06:00
Dane Everitt
355c10c1e4 Update lockfile 2022-02-13 13:17:26 -05:00
Dane Everitt
04933c153b Merge branch 'develop' into v2 2022-02-13 13:10:36 -05:00
Matthew Penner
f1344f1a82 Merge branch 'develop' into v2 2022-01-18 13:12:09 -07:00
Matthew Penner
d874af85db tweaks to websocket event handling 2021-11-15 10:13:31 -07:00
Matthew Penner
54b6033392 update dependencies 2021-11-15 10:13:19 -07:00
Matthew Penner
10a2ffc0a7 Merge branch 'develop' into v2 2021-10-28 23:58:12 -06:00
Matthew Penner
ede1cdc76f Merge branch 'develop' into v2 2021-10-05 19:01:34 -06:00
Dane Everitt
9e98287172 Merge branch 'develop' into v2 2021-09-19 11:17:05 -07:00
Matthew Penner
6653466ca8 Merge branch 'develop' into v2 2021-09-01 16:33:32 -06:00
Matthew Penner
f54a736353 add support for systemd notify 2021-08-02 15:17:22 -06:00
Matthew Penner
d3360f0fd9 sftp: add support for ssh keys 2021-08-02 15:17:22 -06:00
59 changed files with 1158 additions and 1703 deletions

3
.github/FUNDING.yml vendored
View File

@@ -1 +1,2 @@
github: [ matthewpi ] github: [ DaneEveritt ]
custom: [ "https://paypal.me/PterodactylSoftware" ]

View File

@@ -12,7 +12,7 @@ jobs:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ ubuntu-20.04 ] os: [ ubuntu-20.04 ]
go: [ '1.18.7' ] go: [ '^1.17' ]
goos: [ linux ] goos: [ linux ]
goarch: [ amd64, arm64 ] goarch: [ amd64, arm64 ]
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}

View File

@@ -11,7 +11,7 @@ jobs:
uses: actions/checkout@v2 uses: actions/checkout@v2
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: '1.18.7' go-version: '^1.17'
- name: Build - name: Build
env: env:
REF: ${{ github.ref }} REF: ${{ github.ref }}

View File

@@ -1,68 +1,5 @@
# Changelog # Changelog
## v1.7.5
### Fixed
* CVE-2023-32080
## v1.7.4
### Fixed
* CVE-2023-25168
## v1.7.3
### Fixed
* CVE-2023-25152
## v1.7.2
### Fixed
* The S3 backup driver now supports Cloudflare R2
### Added
* During a server transfer, there is a new "Archiving" status that outputs the progress of creating the server transfer archive.
* Adds a configuration option to control the list of trusted proxies that can be used to determine the client IP address.
* Adds a configuration option to control the Docker username space setting when Wings creates containers.
### Changed
* Releases are now built using `Go 1.18` — the minimum version required to build Wings is now `Go 1.18`.
## v1.7.1
### Fixed
* YAML parser has been updated to fix some strange issues
### Added
* Added `Force Outgoing IP` option for servers to ensure outgoing traffic uses the server's IP address
* Adds an option to control the level of gzip compression for backups
## v1.7.0
### Fixed
* Fixes multi-platform support for Wings' Docker image.
### Added
* Adds support for tracking of SFTP actions, power actions, server commands, and file uploads by utilizing a local SQLite database and processing events before sending them to the Panel.
* Adds support for configuring the MTU on the `pterodactyl0` network.
## v1.6.4
### Fixed
* Fixes a bug causing CPU limiting to not be properly applied to servers.
* Fixes a bug causing zip archives to decompress without taking into account nested folder structures.
## v1.6.3
### Fixed
* Fixes SFTP authentication failing for administrative users due to a permissions adjustment on the Panel.
## v1.6.2
### Fixed
* Fixes file upload size not being properly enforced.
* Fixes a bug that prevented listing a directory when it contained a named pipe. Also added a check to prevent attempting to read a named pipe directly.
* Fixes a bug with the archiver logic that would include folders that had the same name prefix. (for example, requesting only `map` would also include `map2` and `map3`)
* Requests to the Panel that return a client error (4xx response code) no longer trigger an exponential backoff, they immediately stop the request.
### Changed
* CPU limit fields are only set on the Docker container if they have been specified for the server — otherwise they are left empty.
### Added
* Added the ability to define the location of the temporary folder used by Wings — defaults to `/tmp/pterodactyl`.
* Adds the ability to authenticate for SFTP using public keys (requires `Panel@1.8.0`).
## v1.6.1 ## v1.6.1
### Fixed ### Fixed
* Fixes error that would sometimes occur when starting a server that would cause the temporary power action lock to never be released due to a blocked channel. * Fixes error that would sometimes occur when starting a server that would cause the temporary power action lock to never be released due to a blocked channel.

View File

@@ -1,18 +1,19 @@
# Stage 1 (Build) # Stage 1 (Build)
FROM golang:1.18-alpine AS builder FROM --platform=$BUILDPLATFORM golang:1.17-alpine AS builder
ARG VERSION ARG VERSION
RUN apk add --update --no-cache git make RUN apk add --update --no-cache git make upx
WORKDIR /app/ WORKDIR /app/
COPY go.mod go.sum /app/ COPY go.mod go.sum /app/
RUN go mod download RUN go mod download
COPY . /app/ COPY . /app/
RUN CGO_ENABLED=0 go build \ RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
-ldflags="-s -w -X github.com/pterodactyl/wings/system.Version=$VERSION" \ -ldflags="-s -w -X github.com/pterodactyl/wings/system.Version=$VERSION" \
-v \ -v \
-trimpath \ -trimpath \
-o wings \ -o wings \
wings.go wings.go
RUN upx wings
RUN echo "ID=\"distroless\"" > /etc/os-release RUN echo "ID=\"distroless\"" > /etc/os-release
# Stage 2 (Final) # Stage 2 (Final)

View File

@@ -14,22 +14,27 @@ dependencies, and allowing users to authenticate with the same credentials they
## Sponsors ## Sponsors
I would like to extend my sincere thanks to the following sponsors for helping find Pterodactyl's developement. I would like to extend my sincere thanks to the following sponsors for helping find Pterodactyl's developement.
[Interested in becoming a sponsor?](https://github.com/sponsors/matthewpi) [Interested in becoming a sponsor?](https://github.com/sponsors/DaneEveritt)
| Company | About | | Company | About |
| ------- | ----- | | ------- | ----- |
| [**WISP**](https://wisp.gg) | Extra features. | | [**WISP**](https://wisp.gg) | Extra features. |
| [**MixmlHosting**](https://mixmlhosting.com) | MixmlHosting provides high quality Virtual Private Servers along with game servers, all at a affordable price. |
| [**BisectHosting**](https://www.bisecthosting.com/) | BisectHosting provides Minecraft, Valheim and other server hosting services with the highest reliability and lightning fast support since 2012. | | [**BisectHosting**](https://www.bisecthosting.com/) | BisectHosting provides Minecraft, Valheim and other server hosting services with the highest reliability and lightning fast support since 2012. |
| [**Fragnet**](https://fragnet.net) | Providing low latency, high-end game hosting solutions to gamers, game studios and eSports platforms. |
| [**Tempest**](https://tempest.net/) | Tempest Hosting is a subsidiary of Path Network, Inc. offering unmetered DDoS protected 10Gbps dedicated servers, starting at just $80/month. Full anycast, tons of filters. |
| [**Bloom.host**](https://bloom.host) | Bloom.host offers dedicated core VPS and Minecraft hosting with Ryzen 9 processors. With owned-hardware, we offer truly unbeatable prices on high-performance hosting. | | [**Bloom.host**](https://bloom.host) | Bloom.host offers dedicated core VPS and Minecraft hosting with Ryzen 9 processors. With owned-hardware, we offer truly unbeatable prices on high-performance hosting. |
| [**MineStrator**](https://minestrator.com/) | Looking for the most highend French hosting company for your minecraft server? More than 24,000 members on our discord trust us. Give us a try! | | [**MineStrator**](https://minestrator.com/) | Looking for a French highend hosting company for you minecraft server? More than 14,000 members on our discord, trust us. |
| [**DedicatedMC**](https://dedicatedmc.io/) | DedicatedMC provides Raw Power hosting at affordable pricing, making sure to never compromise on your performance and giving you the best performance money can buy. |
| [**Skynode**](https://www.skynode.pro/) | Skynode provides blazing fast game servers along with a top-notch user experience. Whatever our clients are looking for, we're able to provide it! | | [**Skynode**](https://www.skynode.pro/) | Skynode provides blazing fast game servers along with a top-notch user experience. Whatever our clients are looking for, we're able to provide it! |
| [**XCORE**](https://xcore-server.de/) | XCORE offers High-End Servers for hosting and gaming since 2012. Fast, excellent and well-known for eSports Gaming. |
| [**RoyaleHosting**](https://royalehosting.net/) | Build your dreams and deploy them with RoyaleHostings reliable servers and network. Easy to use, provisioned in a couple of minutes. |
| [**Spill Hosting**](https://spillhosting.no/) | Spill Hosting is a Norwegian hosting service, which aims for inexpensive services on quality servers. Premium i9-9900K processors will run your game like a dream. |
| [**DeinServerHost**](https://deinserverhost.de/) | DeinServerHost offers Dedicated, vps and Gameservers for many popular Games like Minecraft and Rust in Germany since 2013. | | [**DeinServerHost**](https://deinserverhost.de/) | DeinServerHost offers Dedicated, vps and Gameservers for many popular Games like Minecraft and Rust in Germany since 2013. |
| [**HostBend**](https://hostbend.com/) | HostBend offers a variety of solutions for developers, students, and others who have a tight budget but don't want to compromise quality and support. |
| [**Capitol Hosting Solutions**](https://chs.gg/) | CHS is *the* budget friendly hosting company for Australian and American gamers, offering a variety of plans from Web Hosting to Game Servers; Custom Solutions too! |
| [**ByteAnia**](https://byteania.com/?utm_source=pterodactyl) | ByteAnia offers the best performing and most affordable **Ryzen 5000 Series hosting** on the market for *unbeatable prices*! |
| [**Aussie Server Hosts**](https://aussieserverhosts.com/) | No frills Australian Owned and operated High Performance Server hosting for some of the most demanding games serving Australia and New Zealand. | | [**Aussie Server Hosts**](https://aussieserverhosts.com/) | No frills Australian Owned and operated High Performance Server hosting for some of the most demanding games serving Australia and New Zealand. |
| [**HostEZ**](https://hostez.io) | Providing North America Valheim, Minecraft and other popular games with low latency, high uptime and maximum availability. EZ! |
| [**VibeGAMES**](https://vibegames.net/) | VibeGAMES is a game server provider that specializes in DDOS protection for the games we offer. We have multiple locations in the US, Brazil, France, Germany, Singapore, Australia and South Africa.| | [**VibeGAMES**](https://vibegames.net/) | VibeGAMES is a game server provider that specializes in DDOS protection for the games we offer. We have multiple locations in the US, Brazil, France, Germany, Singapore, Australia and South Africa.|
| [**Gamenodes**](https://gamenodes.nl) | Gamenodes love quality. For Minecraft, Discord Bots and other services, among others. With our own programmers, we provide just that little bit of extra service! | | [**RocketNode**](https://rocketnode.net) | RocketNode is a VPS and Game Server provider that offers the best performing VPS and Game hosting Solutions at affordable prices! |
## Documentation ## Documentation
* [Panel Documentation](https://pterodactyl.io/panel/1.0/getting_started.html) * [Panel Documentation](https://pterodactyl.io/panel/1.0/getting_started.html)

View File

@@ -58,7 +58,7 @@ func newDiagnosticsCommand() *cobra.Command {
return command return command
} }
// diagnosticsCmdRun collects diagnostics about wings, its configuration and the node. // diagnosticsCmdRun collects diagnostics about wings, it's configuration and the node.
// We collect: // We collect:
// - wings and docker versions // - wings and docker versions
// - relevant parts of daemon configuration // - relevant parts of daemon configuration

View File

@@ -5,17 +5,17 @@ import (
"crypto/tls" "crypto/tls"
"errors" "errors"
"fmt" "fmt"
"github.com/pterodactyl/wings/internal/cron"
"github.com/pterodactyl/wings/internal/database"
log2 "log" log2 "log"
"net/http" "net/http"
_ "net/http/pprof" _ "net/http/pprof"
"os" "os"
"os/signal"
"path" "path"
"path/filepath" "path/filepath"
"runtime" "runtime"
"strconv" "strconv"
"strings" "strings"
"syscall"
"time" "time"
"github.com/NYTimes/logrotate" "github.com/NYTimes/logrotate"
@@ -30,6 +30,7 @@ import (
"github.com/pterodactyl/wings/config" "github.com/pterodactyl/wings/config"
"github.com/pterodactyl/wings/environment" "github.com/pterodactyl/wings/environment"
"github.com/pterodactyl/wings/internal/notify"
"github.com/pterodactyl/wings/loggers/cli" "github.com/pterodactyl/wings/loggers/cli"
"github.com/pterodactyl/wings/remote" "github.com/pterodactyl/wings/remote"
"github.com/pterodactyl/wings/router" "github.com/pterodactyl/wings/router"
@@ -81,7 +82,7 @@ func init() {
rootCommand.Flags().Bool("pprof", false, "if the pprof profiler should be enabled. The profiler will bind to localhost:6060 by default") rootCommand.Flags().Bool("pprof", false, "if the pprof profiler should be enabled. The profiler will bind to localhost:6060 by default")
rootCommand.Flags().Int("pprof-block-rate", 0, "enables block profile support, may have performance impacts") rootCommand.Flags().Int("pprof-block-rate", 0, "enables block profile support, may have performance impacts")
rootCommand.Flags().Int("pprof-port", 6060, "If provided with --pprof, the port it will run on") rootCommand.Flags().Int("pprof-port", 6060, "If provided with --pprof, the port it will run on")
rootCommand.Flags().Bool("auto-tls", false, "pass in order to have wings generate and manage its own SSL certificates using Let's Encrypt") rootCommand.Flags().Bool("auto-tls", false, "pass in order to have wings generate and manage it's own SSL certificates using Let's Encrypt")
rootCommand.Flags().String("tls-hostname", "", "required with --auto-tls, the FQDN for the generated SSL certificate") rootCommand.Flags().String("tls-hostname", "", "required with --auto-tls, the FQDN for the generated SSL certificate")
rootCommand.Flags().Bool("ignore-certificate-errors", false, "ignore certificate verification errors when executing API calls") rootCommand.Flags().Bool("ignore-certificate-errors", false, "ignore certificate verification errors when executing API calls")
@@ -132,10 +133,6 @@ func rootCmdRun(cmd *cobra.Command, _ []string) {
}), }),
) )
if err := database.Initialize(); err != nil {
log.WithField("error", err).Fatal("failed to initialize database")
}
manager, err := server.NewManager(cmd.Context(), pclient) manager, err := server.NewManager(cmd.Context(), pclient)
if err != nil { if err != nil {
log.WithField("error", err).Fatal("failed to load server configurations") log.WithField("error", err).Fatal("failed to load server configurations")
@@ -162,7 +159,7 @@ func rootCmdRun(cmd *cobra.Command, _ []string) {
ticker := time.NewTicker(time.Minute) ticker := time.NewTicker(time.Minute)
// Every minute, write the current server states to the disk to allow for a more // Every minute, write the current server states to the disk to allow for a more
// seamless hard-reboot process in which wings will re-sync server states based // seamless hard-reboot process in which wings will re-sync server states based
// on its last tracked state. // on it's last tracked state.
go func() { go func() {
for { for {
select { select {
@@ -265,13 +262,6 @@ func rootCmdRun(cmd *cobra.Command, _ []string) {
} }
}() }()
if s, err := cron.Scheduler(cmd.Context(), manager); err != nil {
log.WithField("error", err).Fatal("failed to initialize cron system")
} else {
log.WithField("subsystem", "cron").Info("starting cron processes")
s.StartAsync()
}
go func() { go func() {
// Run the SFTP server. // Run the SFTP server.
if err := sftp.New(manager).Run(); err != nil { if err := sftp.New(manager).Run(); err != nil {
@@ -337,43 +327,57 @@ func rootCmdRun(cmd *cobra.Command, _ []string) {
} }
// Check if the server should run with TLS but using autocert. // Check if the server should run with TLS but using autocert.
if autotls { go func(s *http.Server, api config.ApiConfiguration, sys config.SystemConfiguration, autotls bool, tlshostname string) {
m := autocert.Manager{ if autotls {
Prompt: autocert.AcceptTOS, m := autocert.Manager{
Cache: autocert.DirCache(path.Join(sys.RootDirectory, "/.tls-cache")), Prompt: autocert.AcceptTOS,
HostPolicy: autocert.HostWhitelist(tlshostname), Cache: autocert.DirCache(path.Join(sys.RootDirectory, "/.tls-cache")),
} HostPolicy: autocert.HostWhitelist(tlshostname),
log.WithField("hostname", tlshostname).Info("webserver is now listening with auto-TLS enabled; certificates will be automatically generated by Let's Encrypt")
// Hook autocert into the main http server.
s.TLSConfig.GetCertificate = m.GetCertificate
s.TLSConfig.NextProtos = append(s.TLSConfig.NextProtos, acme.ALPNProto) // enable tls-alpn ACME challenges
// Start the autocert server.
go func() {
if err := http.ListenAndServe(":http", m.HTTPHandler(nil)); err != nil {
log.WithError(err).Error("failed to serve autocert http server")
} }
}()
// Start the main http server with TLS using autocert. log.WithField("hostname", tlshostname).Info("webserver is now listening with auto-TLS enabled; certificates will be automatically generated by Let's Encrypt")
if err := s.ListenAndServeTLS("", ""); err != nil {
log.WithFields(log.Fields{"auto_tls": true, "tls_hostname": tlshostname, "error": err}).Fatal("failed to configure HTTP server using auto-tls") // Hook autocert into the main http server.
s.TLSConfig.GetCertificate = m.GetCertificate
s.TLSConfig.NextProtos = append(s.TLSConfig.NextProtos, acme.ALPNProto) // enable tls-alpn ACME challenges
// Start the autocert server.
go func() {
if err := http.ListenAndServe(":http", m.HTTPHandler(nil)); err != nil {
log.WithError(err).Error("failed to serve autocert http server")
}
}()
// Start the main http server with TLS using autocert.
if err := s.ListenAndServeTLS("", ""); err != nil {
log.WithFields(log.Fields{"auto_tls": true, "tls_hostname": tlshostname, "error": err}).Fatal("failed to configure HTTP server using auto-tls")
}
return
} }
return
// Check if main http server should run with TLS. Otherwise reset the TLS
// config on the server and then serve it over normal HTTP.
if api.Ssl.Enabled {
if err := s.ListenAndServeTLS(api.Ssl.CertificateFile, api.Ssl.KeyFile); err != nil {
log.WithFields(log.Fields{"auto_tls": false, "error": err}).Fatal("failed to configure HTTPS server")
}
return
}
s.TLSConfig = nil
if err := s.ListenAndServe(); err != nil {
log.WithField("error", err).Fatal("failed to configure HTTP server")
}
}(s, api, sys, autotls, tlshostname)
if err := notify.Readiness(); err != nil {
log.WithField("error", err).Error("failed to notify systemd of readiness state")
} }
// Check if main http server should run with TLS. Otherwise reset the TLS c := make(chan os.Signal, 1)
// config on the server and then serve it over normal HTTP. signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
if api.Ssl.Enabled { <-c
if err := s.ListenAndServeTLS(api.Ssl.CertificateFile, api.Ssl.KeyFile); err != nil {
log.WithFields(log.Fields{"auto_tls": false, "error": err}).Fatal("failed to configure HTTPS server") if err := notify.Stopping(); err != nil {
} log.WithField("error", err).Error("failed to notify systemd of stopping state")
return
}
s.TLSConfig = nil
if err := s.ListenAndServe(); err != nil {
log.WithField("error", err).Fatal("failed to configure HTTP server")
} }
} }

View File

@@ -91,9 +91,6 @@ type ApiConfiguration struct {
// The maximum size for files uploaded through the Panel in MB. // The maximum size for files uploaded through the Panel in MB.
UploadLimit int64 `default:"100" json:"upload_limit" yaml:"upload_limit"` UploadLimit int64 `default:"100" json:"upload_limit" yaml:"upload_limit"`
// A list of IP address of proxies that may send a X-Forwarded-For header to set the true clients IP
TrustedProxies []string `json:"trusted_proxies" yaml:"trusted_proxies"`
} }
// RemoteQueryConfiguration defines the configuration settings for remote requests // RemoteQueryConfiguration defines the configuration settings for remote requests
@@ -166,15 +163,6 @@ type SystemConfiguration struct {
// disk usage is not a concern. // disk usage is not a concern.
DiskCheckInterval int64 `default:"150" yaml:"disk_check_interval"` DiskCheckInterval int64 `default:"150" yaml:"disk_check_interval"`
// ActivitySendInterval is the amount of time that should ellapse between aggregated server activity
// being sent to the Panel. By default this will send activity collected over the last minute. Keep
// in mind that only a fixed number of activity log entries, defined by ActivitySendCount, will be sent
// in each run.
ActivitySendInterval int `default:"60" yaml:"activity_send_interval"`
// ActivitySendCount is the number of activity events to send per batch.
ActivitySendCount int `default:"100" yaml:"activity_send_count"`
// If set to true, file permissions for a server will be checked when the process is // If set to true, file permissions for a server will be checked when the process is
// booted. This can cause boot delays if the server has a large amount of files. In most // booted. This can cause boot delays if the server has a large amount of files. In most
// cases disabling this should not have any major impact unless external processes are // cases disabling this should not have any major impact unless external processes are
@@ -222,15 +210,6 @@ type Backups struct {
// //
// Defaults to 0 (unlimited) // Defaults to 0 (unlimited)
WriteLimit int `default:"0" yaml:"write_limit"` WriteLimit int `default:"0" yaml:"write_limit"`
// CompressionLevel determines how much backups created by wings should be compressed.
//
// "none" -> no compression will be applied
// "best_speed" -> uses gzip level 1 for fast speed
// "best_compression" -> uses gzip level 9 for minimal disk space useage
//
// Defaults to "best_speed" (level 1)
CompressionLevel string `default:"best_speed" yaml:"compression_level"`
} }
type Transfers struct { type Transfers struct {

View File

@@ -36,7 +36,6 @@ type DockerNetworkConfiguration struct {
Mode string `default:"pterodactyl_nw" yaml:"network_mode"` Mode string `default:"pterodactyl_nw" yaml:"network_mode"`
IsInternal bool `default:"false" yaml:"is_internal"` IsInternal bool `default:"false" yaml:"is_internal"`
EnableICC bool `default:"true" yaml:"enable_icc"` EnableICC bool `default:"true" yaml:"enable_icc"`
NetworkMTU int64 `default:"1500" yaml:"network_mtu"`
Interfaces dockerNetworkInterfaces `yaml:"interfaces"` Interfaces dockerNetworkInterfaces `yaml:"interfaces"`
} }
@@ -78,14 +77,6 @@ type DockerConfiguration struct {
Overhead Overhead `json:"overhead" yaml:"overhead"` Overhead Overhead `json:"overhead" yaml:"overhead"`
UsePerformantInspect bool `default:"true" json:"use_performant_inspect" yaml:"use_performant_inspect"` UsePerformantInspect bool `default:"true" json:"use_performant_inspect" yaml:"use_performant_inspect"`
// Sets the user namespace mode for the container when user namespace remapping option is
// enabled.
//
// If the value is blank, the daemon's user namespace remapping configuration is used,
// if the value is "host", then the pterodactyl containers are started with user namespace
// remapping disabled
UsernsMode string `default:"" json:"userns_mode" yaml:"userns_mode"`
} }
// RegistryConfiguration defines the authentication credentials for a given // RegistryConfiguration defines the authentication credentials for a given

View File

@@ -12,11 +12,6 @@ import (
// Defines the allocations available for a given server. When using the Docker environment // Defines the allocations available for a given server. When using the Docker environment
// driver these correspond to mappings for the container that allow external connections. // driver these correspond to mappings for the container that allow external connections.
type Allocations struct { type Allocations struct {
// ForceOutgoingIP causes a dedicated bridge network to be created for the
// server with a special option, causing Docker to SNAT outgoing traffic to
// the DefaultMapping's IP. This is important to servers which rely on external
// services that check the IP of the server (Source Engine servers, for example).
ForceOutgoingIP bool `json:"force_outgoing_ip"`
// Defines the default allocation that should be used for this server. This is // Defines the default allocation that should be used for this server. This is
// what will be used for {SERVER_IP} and {SERVER_PORT} when modifying configuration // what will be used for {SERVER_IP} and {SERVER_PORT} when modifying configuration
// files or the startup arguments for a server. // files or the startup arguments for a server.

View File

@@ -41,12 +41,12 @@ func ConfigureDocker(ctx context.Context) error {
nw := config.Get().Docker.Network nw := config.Get().Docker.Network
resource, err := cli.NetworkInspect(ctx, nw.Name, types.NetworkInspectOptions{}) resource, err := cli.NetworkInspect(ctx, nw.Name, types.NetworkInspectOptions{})
if err != nil { if err != nil {
if !client.IsErrNotFound(err) { if client.IsErrNotFound(err) {
return err log.Info("creating missing pterodactyl0 interface, this could take a few seconds...")
} if err := createDockerNetwork(ctx, cli); err != nil {
return err
log.Info("creating missing pterodactyl0 interface, this could take a few seconds...") }
if err := createDockerNetwork(ctx, cli); err != nil { } else {
return err return err
} }
} }
@@ -92,7 +92,7 @@ func createDockerNetwork(ctx context.Context, cli *client.Client) error {
"com.docker.network.bridge.enable_ip_masquerade": "true", "com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0", "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "pterodactyl0", "com.docker.network.bridge.name": "pterodactyl0",
"com.docker.network.driver.mtu": strconv.FormatInt(nw.NetworkMTU, 10), "com.docker.network.driver.mtu": "1500",
}, },
}) })
if err != nil { if err != nil {

View File

@@ -147,12 +147,10 @@ func (e *Environment) InSituUpdate() error {
// currently available for it. If the container already exists it will be // currently available for it. If the container already exists it will be
// returned. // returned.
func (e *Environment) Create() error { func (e *Environment) Create() error {
ctx := context.Background()
// If the container already exists don't hit the user with an error, just return // If the container already exists don't hit the user with an error, just return
// the current information about it which is what we would do when creating the // the current information about it which is what we would do when creating the
// container anyways. // container anyways.
if _, err := e.ContainerInspect(ctx); err == nil { if _, err := e.ContainerInspect(context.Background()); err == nil {
return nil return nil
} else if !client.IsErrNotFound(err) { } else if !client.IsErrNotFound(err) {
return errors.Wrap(err, "environment/docker: failed to inspect container") return errors.Wrap(err, "environment/docker: failed to inspect container")
@@ -192,34 +190,7 @@ func (e *Environment) Create() error {
}, },
} }
networkMode := container.NetworkMode(config.Get().Docker.Network.Mode) tmpfsSize := strconv.Itoa(int(config.Get().Docker.TmpfsSize))
if a.ForceOutgoingIP {
e.log().Debug("environment/docker: forcing outgoing IP address")
networkName := strings.ReplaceAll(e.Id, "-", "")
networkMode = container.NetworkMode(networkName)
if _, err := e.client.NetworkInspect(ctx, networkName, types.NetworkInspectOptions{}); err != nil {
if !client.IsErrNotFound(err) {
return err
}
if _, err := e.client.NetworkCreate(ctx, networkName, types.NetworkCreate{
Driver: "bridge",
EnableIPv6: false,
Internal: false,
Attachable: false,
Ingress: false,
ConfigOnly: false,
Options: map[string]string{
"encryption": "false",
"com.docker.network.bridge.default_bridge": "false",
"com.docker.network.host_ipv4": a.DefaultMapping.Ip,
},
}); err != nil {
return err
}
}
}
hostConf := &container.HostConfig{ hostConf := &container.HostConfig{
PortBindings: a.DockerBindings(), PortBindings: a.DockerBindings(),
@@ -231,7 +202,7 @@ func (e *Environment) Create() error {
// Configure the /tmp folder mapping in containers. This is necessary for some // Configure the /tmp folder mapping in containers. This is necessary for some
// games that need to make use of it for downloads and other installation processes. // games that need to make use of it for downloads and other installation processes.
Tmpfs: map[string]string{ Tmpfs: map[string]string{
"/tmp": "rw,exec,nosuid,size=" + strconv.Itoa(int(config.Get().Docker.TmpfsSize)) + "M", "/tmp": "rw,exec,nosuid,size=" + tmpfsSize + "M",
}, },
// Define resource limits for the container based on the data passed through // Define resource limits for the container based on the data passed through
@@ -260,11 +231,10 @@ func (e *Environment) Create() error {
"setpcap", "mknod", "audit_write", "net_raw", "dac_override", "setpcap", "mknod", "audit_write", "net_raw", "dac_override",
"fowner", "fsetid", "net_bind_service", "sys_chroot", "setfcap", "fowner", "fsetid", "net_bind_service", "sys_chroot", "setfcap",
}, },
NetworkMode: networkMode, NetworkMode: container.NetworkMode(config.Get().Docker.Network.Mode),
UsernsMode: container.UsernsMode(config.Get().Docker.UsernsMode),
} }
if _, err := e.client.ContainerCreate(ctx, conf, hostConf, nil, nil, e.Id); err != nil { if _, err := e.client.ContainerCreate(context.Background(), conf, hostConf, nil, nil, e.Id); err != nil {
return errors.Wrap(err, "environment/docker: failed to create container") return errors.Wrap(err, "environment/docker: failed to create container")
} }
@@ -510,3 +480,21 @@ func (e *Environment) convertMounts() []mount.Mount {
return out return out
} }
func (e *Environment) resources() container.Resources {
l := e.Configuration.Limits()
pids := l.ProcessLimit()
return container.Resources{
Memory: l.BoundedMemoryLimit(),
MemoryReservation: l.MemoryLimit * 1_000_000,
MemorySwap: l.ConvertedSwap(),
CPUQuota: l.ConvertedCpuLimit(),
CPUPeriod: 100_000,
CPUShares: 1024,
BlkioWeight: l.IoWeight,
OomKillDisable: &l.OOMDisabled,
CpusetCpus: l.Threads,
PidsLimit: &pids,
}
}

View File

@@ -99,36 +99,21 @@ func (l Limits) ProcessLimit() int64 {
return config.Get().Docker.ContainerPidLimit return config.Get().Docker.ContainerPidLimit
} }
// AsContainerResources returns the available resources for a container in a format
// that Docker understands.
func (l Limits) AsContainerResources() container.Resources { func (l Limits) AsContainerResources() container.Resources {
pids := l.ProcessLimit() pids := l.ProcessLimit()
resources := container.Resources{
return container.Resources{
Memory: l.BoundedMemoryLimit(), Memory: l.BoundedMemoryLimit(),
MemoryReservation: l.MemoryLimit * 1_000_000, MemoryReservation: l.MemoryLimit * 1_000_000,
MemorySwap: l.ConvertedSwap(), MemorySwap: l.ConvertedSwap(),
CPUQuota: l.ConvertedCpuLimit(),
CPUPeriod: 100_000,
CPUShares: 1024,
BlkioWeight: l.IoWeight, BlkioWeight: l.IoWeight,
OomKillDisable: &l.OOMDisabled, OomKillDisable: &l.OOMDisabled,
CpusetCpus: l.Threads,
PidsLimit: &pids, PidsLimit: &pids,
} }
// If the CPU Limit is not set, don't send any of these fields through. Providing
// them seems to break some Java services that try to read the available processors.
//
// @see https://github.com/pterodactyl/panel/issues/3988
if l.CpuLimit > 0 {
resources.CPUQuota = l.CpuLimit * 1_000
resources.CPUPeriod = 100_000
resources.CPUShares = 1024
}
// Similar to above, don't set the specific assigned CPUs if we didn't actually limit
// the server to any of them.
if l.Threads != "" {
resources.CpusetCpus = l.Threads
}
return resources
} }
type Variables map[string]interface{} type Variables map[string]interface{}

111
go.mod
View File

@@ -1,124 +1,115 @@
module github.com/pterodactyl/wings module github.com/pterodactyl/wings
go 1.18 go 1.17
require ( require (
emperror.dev/errors v0.8.1 emperror.dev/errors v0.8.1
github.com/AlecAivazis/survey/v2 v2.3.6 github.com/AlecAivazis/survey/v2 v2.3.4
github.com/Jeffail/gabs/v2 v2.6.1 github.com/Jeffail/gabs/v2 v2.6.1
github.com/NYTimes/logrotate v1.0.0 github.com/NYTimes/logrotate v1.0.0
github.com/apex/log v1.9.0 github.com/apex/log v1.9.0
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d
github.com/beevik/etree v1.1.0 github.com/beevik/etree v1.1.0
github.com/buger/jsonparser v1.1.1 github.com/buger/jsonparser v1.1.1
github.com/cenkalti/backoff/v4 v4.1.3 github.com/cenkalti/backoff/v4 v4.1.2
github.com/cobaugh/osrelease v0.0.0-20181218015638-a93a0a55a249 github.com/cobaugh/osrelease v0.0.0-20181218015638-a93a0a55a249
github.com/creasty/defaults v1.6.0 github.com/creasty/defaults v1.5.2
github.com/docker/docker v20.10.18+incompatible github.com/docker/docker v20.10.14+incompatible
github.com/docker/go-connections v0.4.0 github.com/docker/go-connections v0.4.0
github.com/fatih/color v1.13.0 github.com/fatih/color v1.13.0
github.com/franela/goblin v0.0.0-20200825194134-80c0062ed6cd github.com/franela/goblin v0.0.0-20200825194134-80c0062ed6cd
github.com/gabriel-vasile/mimetype v1.4.1 github.com/gabriel-vasile/mimetype v1.4.0
github.com/gammazero/workerpool v1.1.3 github.com/gammazero/workerpool v1.1.2
github.com/gbrlsnchs/jwt/v3 v3.0.1 github.com/gbrlsnchs/jwt/v3 v3.0.1
github.com/gin-gonic/gin v1.8.1 github.com/gin-gonic/gin v1.7.7
github.com/glebarez/sqlite v1.4.8
github.com/go-co-op/gocron v1.17.0
github.com/goccy/go-json v0.9.11
github.com/google/uuid v1.3.0 github.com/google/uuid v1.3.0
github.com/gorilla/websocket v1.5.0 github.com/gorilla/websocket v1.5.0
github.com/iancoleman/strcase v0.2.0 github.com/iancoleman/strcase v0.2.0
github.com/icza/dyno v0.0.0-20220812133438-f0b6f8a18845 github.com/icza/dyno v0.0.0-20210726202311-f1bafe5d9996
github.com/juju/ratelimit v1.0.2 github.com/juju/ratelimit v1.0.1
github.com/karrick/godirwalk v1.17.0 github.com/karrick/godirwalk v1.16.1
github.com/klauspost/compress v1.15.11
github.com/klauspost/pgzip v1.2.5 github.com/klauspost/pgzip v1.2.5
github.com/magiconair/properties v1.8.6 github.com/magiconair/properties v1.8.6
github.com/mattn/go-colorable v0.1.13 github.com/mattn/go-colorable v0.1.12
github.com/mholt/archiver/v3 v3.5.1 github.com/mholt/archiver/v3 v3.5.1
github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db
github.com/patrickmn/go-cache v2.1.0+incompatible github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pkg/sftp v1.13.5 github.com/pkg/sftp v1.13.4
github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06 github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06
github.com/spf13/cobra v1.5.0 github.com/spf13/cobra v1.4.0
github.com/stretchr/testify v1.8.0 github.com/stretchr/testify v1.7.0
golang.org/x/crypto v0.0.0-20220926161630-eccd6366d1be golang.org/x/crypto v0.0.0-20220321153916-2c7772ba3064
golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0 golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
gopkg.in/ini.v1 v1.67.0 gopkg.in/ini.v1 v1.66.4
gopkg.in/yaml.v2 v2.4.0 gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1
gorm.io/gorm v1.23.10
) )
require github.com/goccy/go-json v0.9.6
require golang.org/x/sys v0.0.0-20220319134239-a9b59b0215f8 // indirect
require ( require (
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/Microsoft/hcsshim v0.9.4 // indirect github.com/Microsoft/hcsshim v0.9.2 // indirect
github.com/andybalholm/brotli v1.0.4 // indirect github.com/andybalholm/brotli v1.0.4 // indirect
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/containerd/containerd v1.6.2 // indirect
github.com/containerd/fifo v1.0.0 // indirect github.com/containerd/fifo v1.0.0 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect github.com/davecgh/go-spew v1.1.1 // indirect
github.com/docker/distribution v2.8.1+incompatible // indirect github.com/docker/distribution v2.8.1+incompatible // indirect
github.com/docker/go-metrics v0.0.1 // indirect github.com/docker/go-metrics v0.0.1 // indirect
github.com/docker/go-units v0.5.0 // indirect github.com/docker/go-units v0.4.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect github.com/fsnotify/fsnotify v1.5.1 // indirect
github.com/gammazero/deque v0.2.0 // indirect github.com/gammazero/deque v0.1.1 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect github.com/gin-contrib/sse v0.1.0 // indirect
github.com/glebarez/go-sqlite v1.19.1 // indirect
github.com/go-playground/locales v0.14.0 // indirect github.com/go-playground/locales v0.14.0 // indirect
github.com/go-playground/universal-translator v0.18.0 // indirect github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.11.1 // indirect github.com/go-playground/validator/v10 v10.10.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/protobuf v1.5.2 // indirect github.com/golang/protobuf v1.5.2 // indirect
github.com/golang/snappy v0.0.4 // indirect github.com/golang/snappy v0.0.4 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect github.com/gorilla/mux v1.7.4 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/json-iterator/go v1.1.12 // indirect github.com/json-iterator/go v1.1.12 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/compress v1.15.1 // indirect
github.com/kr/fs v0.1.0 // indirect github.com/kr/fs v0.1.0 // indirect
github.com/leodido/go-urn v1.2.1 // indirect github.com/leodido/go-urn v1.2.1 // indirect
github.com/magefile/mage v1.14.0 // indirect github.com/magefile/mage v1.13.0 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect github.com/mattn/go-isatty v0.0.14 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.2 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/moby/term v0.0.0-20220808134915-39b0c02b01ae // indirect github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect github.com/morikuni/aec v1.0.0 // indirect
github.com/nwaples/rardecode v1.1.3 // indirect github.com/nwaples/rardecode v1.1.3 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc2 // indirect github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.0.5 // indirect github.com/pierrec/lz4/v4 v4.1.14 // indirect
github.com/pierrec/lz4/v4 v4.1.17 // indirect
github.com/pkg/errors v0.9.1 // indirect github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_golang v1.13.0 // indirect github.com/prometheus/client_golang v1.12.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.37.0 // indirect github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.8.0 // indirect github.com/prometheus/procfs v0.7.3 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20220927061507-ef77025ab5aa // indirect github.com/sirupsen/logrus v1.8.1 // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/sirupsen/logrus v1.9.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect github.com/spf13/pflag v1.0.5 // indirect
github.com/ugorji/go/codec v1.2.7 // indirect github.com/ugorji/go/codec v1.2.7 // indirect
github.com/ulikunitz/xz v0.5.10 // indirect github.com/ulikunitz/xz v0.5.10 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
go.uber.org/atomic v1.10.0 // indirect go.uber.org/atomic v1.9.0 // indirect
go.uber.org/multierr v1.8.0 // indirect go.uber.org/multierr v1.8.0 // indirect
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4 // indirect golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
golang.org/x/net v0.0.0-20221004154528-8021a29435af // indirect golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect
golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec // indirect
golang.org/x/term v0.0.0-20220919170432-7a66f970e087 // indirect
golang.org/x/text v0.3.7 // indirect golang.org/x/text v0.3.7 // indirect
golang.org/x/time v0.0.0-20220922220347-f3bd1da661af // indirect golang.org/x/time v0.0.0-20220224211638-0e9765cccd65 // indirect
golang.org/x/tools v0.1.12 // indirect golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect google.golang.org/genproto v0.0.0-20220324131243-acbaeb5b85eb // indirect
google.golang.org/protobuf v1.28.1 // indirect google.golang.org/grpc v1.45.0 // indirect
modernc.org/libc v1.20.0 // indirect google.golang.org/protobuf v1.28.0 // indirect
modernc.org/mathutil v1.5.0 // indirect gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
modernc.org/memory v1.4.0 // indirect
modernc.org/sqlite v1.19.1 // indirect
) )

547
go.sum

File diff suppressed because it is too large Load Diff

View File

@@ -1,57 +0,0 @@
package cron
import (
"context"
"emperror.dev/errors"
"github.com/pterodactyl/wings/internal/database"
"github.com/pterodactyl/wings/internal/models"
"github.com/pterodactyl/wings/server"
"github.com/pterodactyl/wings/system"
)
type activityCron struct {
mu *system.AtomicBool
manager *server.Manager
max int
}
// Run executes the cronjob and ensures we fetch and send all of the stored activity to the
// Panel instance. Once activity is sent it is deleted from the local database instance. Any
// SFTP specific events are not handled in this cron, they're handled seperately to account
// for de-duplication and event merging.
func (ac *activityCron) Run(ctx context.Context) error {
// Don't execute this cron if there is currently one running. Once this task is completed
// go ahead and mark it as no longer running.
if !ac.mu.SwapIf(true) {
return errors.WithStack(ErrCronRunning)
}
defer ac.mu.Store(false)
var activity []models.Activity
tx := database.Instance().WithContext(ctx).
Where("event NOT LIKE ?", "server:sftp.%").
Limit(ac.max).
Find(&activity)
if tx.Error != nil {
return errors.WithStack(tx.Error)
}
if len(activity) == 0 {
return nil
}
if err := ac.manager.Client().SendActivityLogs(ctx, activity); err != nil {
return errors.WrapIf(err, "cron: failed to send activity events to Panel")
}
var ids []int
for _, v := range activity {
ids = append(ids, v.ID)
}
tx = database.Instance().WithContext(ctx).Where("id IN ?", ids).Delete(&models.Activity{})
if tx.Error != nil {
return errors.WithStack(tx.Error)
}
return nil
}

View File

@@ -1,71 +0,0 @@
package cron
import (
"context"
"emperror.dev/errors"
log2 "github.com/apex/log"
"github.com/go-co-op/gocron"
"github.com/pterodactyl/wings/config"
"github.com/pterodactyl/wings/server"
"github.com/pterodactyl/wings/system"
"time"
)
const ErrCronRunning = errors.Sentinel("cron: job already running")
var o system.AtomicBool
// Scheduler configures the internal cronjob system for Wings and returns the scheduler
// instance to the caller. This should only be called once per application lifecycle, additional
// calls will result in an error being returned.
func Scheduler(ctx context.Context, m *server.Manager) (*gocron.Scheduler, error) {
if !o.SwapIf(true) {
return nil, errors.New("cron: cannot call scheduler more than once in application lifecycle")
}
l, err := time.LoadLocation(config.Get().System.Timezone)
if err != nil {
return nil, errors.Wrap(err, "cron: failed to parse configured system timezone")
}
activity := activityCron{
mu: system.NewAtomicBool(false),
manager: m,
max: config.Get().System.ActivitySendCount,
}
sftp := sftpCron{
mu: system.NewAtomicBool(false),
manager: m,
max: config.Get().System.ActivitySendCount,
}
s := gocron.NewScheduler(l)
log := log2.WithField("subsystem", "cron")
interval := time.Duration(config.Get().System.ActivitySendInterval) * time.Second
log.WithField("interval", interval).Info("configuring system crons")
_, _ = s.Tag("activity").Every(interval).Do(func() {
log.WithField("cron", "activity").Debug("sending internal activity events to Panel")
if err := activity.Run(ctx); err != nil {
if errors.Is(err, ErrCronRunning) {
log.WithField("cron", "activity").Warn("activity process is already running, skipping...")
} else {
log.WithField("cron", "activity").WithField("error", err).Error("activity process failed to execute")
}
}
})
_, _ = s.Tag("sftp").Every(interval).Do(func() {
log.WithField("cron", "sftp").Debug("sending sftp events to Panel")
if err := sftp.Run(ctx); err != nil {
if errors.Is(err, ErrCronRunning) {
log.WithField("cron", "sftp").Warn("sftp events process already running, skipping...")
} else {
log.WithField("cron", "sftp").WithField("error", err).Error("sftp events process failed to execute")
}
}
})
return s, nil
}

View File

@@ -1,175 +0,0 @@
package cron
import (
"context"
"emperror.dev/errors"
"github.com/pterodactyl/wings/internal/database"
"github.com/pterodactyl/wings/internal/models"
"github.com/pterodactyl/wings/server"
"github.com/pterodactyl/wings/system"
"reflect"
)
type sftpCron struct {
mu *system.AtomicBool
manager *server.Manager
max int
}
type mapKey struct {
User string
Server string
IP string
Event models.Event
Timestamp string
}
type eventMap struct {
max int
ids []int
m map[mapKey]*models.Activity
}
// Run executes the SFTP reconciliation cron. This job will pull all of the SFTP specific events
// and merge them together across user, server, ip, and event. This allows a SFTP event that deletes
// tens or hundreds of files to be tracked as a single "deletion" event so long as they all occur
// within the same one minute period of time (starting at the first timestamp for the group). Without
// this we'd end up flooding the Panel event log with excessive data that is of no use to end users.
func (sc *sftpCron) Run(ctx context.Context) error {
if !sc.mu.SwapIf(true) {
return errors.WithStack(ErrCronRunning)
}
defer sc.mu.Store(false)
var o int
activity, err := sc.fetchRecords(ctx, o)
if err != nil {
return err
}
o += len(activity)
events := &eventMap{
m: map[mapKey]*models.Activity{},
ids: []int{},
max: sc.max,
}
for {
if len(activity) == 0 {
break
}
slen := len(events.ids)
for _, a := range activity {
events.Push(a)
}
if len(events.ids) > slen {
// Execute the query again, we found some events so we want to continue
// with this. Start at the next offset.
activity, err = sc.fetchRecords(ctx, o)
if err != nil {
return errors.WithStack(err)
}
o += len(activity)
} else {
break
}
}
if len(events.m) == 0 {
return nil
}
if err := sc.manager.Client().SendActivityLogs(ctx, events.Elements()); err != nil {
return errors.Wrap(err, "failed to send sftp activity logs to Panel")
}
if tx := database.Instance().Where("id IN ?", events.ids).Delete(&models.Activity{}); tx.Error != nil {
return errors.WithStack(tx.Error)
}
return nil
}
// fetchRecords returns a group of activity events starting at the given offset. This is used
// since we might need to make multiple database queries to select enough events to properly
// fill up our request to the given maximum. This is due to the fact that this cron merges any
// activity that line up across user, server, ip, and event into a single activity record when
// sending the data to the Panel.
func (sc *sftpCron) fetchRecords(ctx context.Context, offset int) (activity []models.Activity, err error) {
tx := database.Instance().WithContext(ctx).
Where("event LIKE ?", "server:sftp.%").
Order("event DESC").
Offset(offset).
Limit(sc.max).
Find(&activity)
if tx.Error != nil {
err = errors.WithStack(tx.Error)
}
return
}
// Push adds an activity to the event mapping, or de-duplicates it and merges the files metadata
// into the existing entity that exists.
func (em *eventMap) Push(a models.Activity) {
m := em.forActivity(a)
// If no activity entity is returned we've hit the cap for the number of events to
// send along to the Panel. Just skip over this record and we'll account for it in
// the next iteration.
if m == nil {
return
}
em.ids = append(em.ids, a.ID)
// Always reduce this to the first timestamp that was recorded for the set
// of events, and not
if a.Timestamp.Before(m.Timestamp) {
m.Timestamp = a.Timestamp
}
list := m.Metadata["files"].([]interface{})
if s, ok := a.Metadata["files"]; ok {
v := reflect.ValueOf(s)
if v.Kind() != reflect.Slice || v.IsNil() {
return
}
for i := 0; i < v.Len(); i++ {
list = append(list, v.Index(i).Interface())
}
// You must set it again at the end of the process, otherwise you've only updated the file
// slice in this one loop since it isn't passed by reference. This is just shorter than having
// to explicitly keep casting it to the slice.
m.Metadata["files"] = list
}
}
// Elements returns the finalized activity models.
func (em *eventMap) Elements() (out []models.Activity) {
for _, v := range em.m {
out = append(out, *v)
}
return
}
// forActivity returns an event entity from our map which allows existing matches to be
// updated with additional files.
func (em *eventMap) forActivity(a models.Activity) *models.Activity {
key := mapKey{
User: a.User.String,
Server: a.Server,
IP: a.IP,
Event: a.Event,
// We group by the minute, don't care about the seconds for this logic.
Timestamp: a.Timestamp.Format("2006-01-02_15:04"),
}
if v, ok := em.m[key]; ok {
return v
}
// Cap the size of the events map at the defined maximum events to send to the Panel. Just
// return nil and let the caller handle it.
if len(em.m) >= em.max {
return nil
}
// Doesn't exist in our map yet, create a copy of the activity passed into this
// function and then assign it into the map with an empty metadata value.
v := a
v.Metadata = models.ActivityMeta{
"files": make([]interface{}, 0),
}
em.m[key] = &v
return &v
}

View File

@@ -1,57 +0,0 @@
package database
import (
"emperror.dev/errors"
"github.com/glebarez/sqlite"
"github.com/pterodactyl/wings/config"
"github.com/pterodactyl/wings/internal/models"
"github.com/pterodactyl/wings/system"
"gorm.io/gorm"
"gorm.io/gorm/logger"
"path/filepath"
"time"
)
var o system.AtomicBool
var db *gorm.DB
// Initialize configures the local SQLite database for Wings and ensures that the models have
// been fully migrated.
func Initialize() error {
if !o.SwapIf(true) {
panic("database: attempt to initialize more than once during application lifecycle")
}
p := filepath.Join(config.Get().System.RootDirectory, "wings.db")
instance, err := gorm.Open(sqlite.Open(p), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
return errors.Wrap(err, "database: could not open database file")
}
db = instance
if sql, err := db.DB(); err != nil {
return errors.WithStack(err)
} else {
sql.SetMaxOpenConns(1)
sql.SetConnMaxLifetime(time.Hour)
}
if tx := db.Exec("PRAGMA synchronous = OFF"); tx.Error != nil {
return errors.WithStack(tx.Error)
}
if tx := db.Exec("PRAGMA journal_mode = MEMORY"); tx.Error != nil {
return errors.WithStack(tx.Error)
}
if err := db.AutoMigrate(&models.Activity{}); err != nil {
return errors.WithStack(err)
}
return nil
}
// Instance returns the gorm database instance that was configured when the application was
// booted.
func Instance() *gorm.DB {
if db == nil {
panic("database: attempt to access instance before initialized")
}
return db
}

View File

@@ -1,67 +0,0 @@
package models
import (
"github.com/pterodactyl/wings/system"
"gorm.io/gorm"
"time"
)
type Event string
type ActivityMeta map[string]interface{}
// Activity defines an activity log event for a server entity performed by a user. This is
// used for tracking commands, power actions, and SFTP events so that they can be reconciled
// and sent back to the Panel instance to be displayed to the user.
type Activity struct {
ID int `gorm:"primaryKey;not null" json:"-"`
// User is UUID of the user that triggered this event, or an empty string if the event
// cannot be tied to a specific user, in which case we will assume it was the system
// user.
User JsonNullString `gorm:"type:uuid" json:"user"`
// Server is the UUID of the server this event is associated with.
Server string `gorm:"type:uuid;not null" json:"server"`
// Event is a string that describes what occurred, and is used by the Panel instance to
// properly associate this event in the activity logs.
Event Event `gorm:"index;not null" json:"event"`
// Metadata is either a null value, string, or a JSON blob with additional event specific
// metadata that can be provided.
Metadata ActivityMeta `gorm:"serializer:json" json:"metadata"`
// IP is the IP address that triggered this event, or an empty string if it cannot be
// determined properly. This should be the connecting user's IP address, and not the
// internal system IP.
IP string `gorm:"not null" json:"ip"`
Timestamp time.Time `gorm:"not null" json:"timestamp"`
}
// SetUser sets the current user that performed the action. If an empty string is provided
// it is cast into a null value when stored.
func (a Activity) SetUser(u string) *Activity {
var ns JsonNullString
if u == "" {
if err := ns.Scan(nil); err != nil {
panic(err)
}
} else {
if err := ns.Scan(u); err != nil {
panic(err)
}
}
a.User = ns
return &a
}
// BeforeCreate executes before we create any activity entry to ensure the IP address
// is trimmed down to remove any extraneous data, and the timestamp is set to the current
// system time and then stored as UTC.
func (a *Activity) BeforeCreate(_ *gorm.DB) error {
a.IP = system.TrimIPSuffix(a.IP)
if a.Timestamp.IsZero() {
a.Timestamp = time.Now()
}
a.Timestamp = a.Timestamp.UTC()
if a.Metadata == nil {
a.Metadata = ActivityMeta{}
}
return nil
}

View File

@@ -1,31 +0,0 @@
package models
import (
"database/sql"
"emperror.dev/errors"
"github.com/goccy/go-json"
)
type JsonNullString struct {
sql.NullString
}
func (v JsonNullString) MarshalJSON() ([]byte, error) {
if v.Valid {
return json.Marshal(v.String)
} else {
return json.Marshal(nil)
}
}
func (v *JsonNullString) UnmarshalJSON(data []byte) error {
var s *string
if err := json.Unmarshal(data, &s); err != nil {
return errors.WithStack(err)
}
if s != nil {
v.String = *s
}
v.Valid = s != nil
return nil
}

19
internal/notify/notify.go Normal file
View File

@@ -0,0 +1,19 @@
// Package notify handles notifying the operating system of the program's state.
//
// For linux based operating systems, this is done through the systemd socket
// set by "NOTIFY_SOCKET" environment variable.
//
// Currently, no other operating systems are supported.
package notify
func Readiness() error {
return readiness()
}
func Reloading() error {
return reloading()
}
func Stopping() error {
return stopping()
}

View File

@@ -0,0 +1,48 @@
package notify
import (
"io"
"net"
"os"
"strings"
)
func notify(path string, r io.Reader) error {
s := &net.UnixAddr{
Name: path,
Net: "unixgram",
}
c, err := net.DialUnix(s.Net, nil, s)
if err != nil {
return err
}
defer c.Close()
if _, err := io.Copy(c, r); err != nil {
return err
}
return nil
}
func socketNotify(payload string) error {
v, ok := os.LookupEnv("NOTIFY_SOCKET")
if !ok || v == "" {
return nil
}
if err := notify(v, strings.NewReader(payload)); err != nil {
return err
}
return nil
}
func readiness() error {
return socketNotify("READY=1")
}
func reloading() error {
return socketNotify("RELOADING=1")
}
func stopping() error {
return socketNotify("STOPPING=1")
}

View File

@@ -0,0 +1,16 @@
//go:build !linux
// +build !linux
package notify
func readiness() error {
return nil
}
func reloading() error {
return nil
}
func stopping() error {
return nil
}

View File

@@ -15,7 +15,7 @@ import (
"github.com/icza/dyno" "github.com/icza/dyno"
"github.com/magiconair/properties" "github.com/magiconair/properties"
"gopkg.in/ini.v1" "gopkg.in/ini.v1"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v2"
"github.com/pterodactyl/wings/config" "github.com/pterodactyl/wings/config"
) )

View File

@@ -4,7 +4,6 @@ import (
"bytes" "bytes"
"context" "context"
"fmt" "fmt"
"github.com/pterodactyl/wings/internal/models"
"io" "io"
"net/http" "net/http"
"strconv" "strconv"
@@ -31,7 +30,6 @@ type Client interface {
SetInstallationStatus(ctx context.Context, uuid string, successful bool) error SetInstallationStatus(ctx context.Context, uuid string, successful bool) error
SetTransferStatus(ctx context.Context, uuid string, successful bool) error SetTransferStatus(ctx context.Context, uuid string, successful bool) error
ValidateSftpCredentials(ctx context.Context, request SftpAuthRequest) (SftpAuthResponse, error) ValidateSftpCredentials(ctx context.Context, request SftpAuthRequest) (SftpAuthResponse, error)
SendActivityLogs(ctx context.Context, activity []models.Activity) error
} }
type client struct { type client struct {
@@ -130,19 +128,10 @@ func (c *client) requestOnce(ctx context.Context, method, path string, body io.R
// and adds the required authentication headers to the request that is being // and adds the required authentication headers to the request that is being
// created. Errors returned will be of the RequestError type if there was some // created. Errors returned will be of the RequestError type if there was some
// type of response from the API that can be parsed. // type of response from the API that can be parsed.
func (c *client) request(ctx context.Context, method, path string, body *bytes.Buffer, opts ...func(r *http.Request)) (*Response, error) { func (c *client) request(ctx context.Context, method, path string, body io.Reader, opts ...func(r *http.Request)) (*Response, error) {
var res *Response var res *Response
err := backoff.Retry(func() error { err := backoff.Retry(func() error {
var b bytes.Buffer r, err := c.requestOnce(ctx, method, path, body, opts...)
if body != nil {
// We have to create a copy of the body, otherwise attempting this request again will
// send no data if there was initially a body since the "requestOnce" method will read
// the whole buffer, thus leaving it empty at the end.
if _, err := b.Write(body.Bytes()); err != nil {
return backoff.Permanent(errors.Wrap(err, "http: failed to copy body buffer"))
}
}
r, err := c.requestOnce(ctx, method, path, &b, opts...)
if err != nil { if err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return backoff.Permanent(err) return backoff.Permanent(err)
@@ -153,10 +142,12 @@ func (c *client) request(ctx context.Context, method, path string, body *bytes.B
if r.HasError() { if r.HasError() {
// Close the request body after returning the error to free up resources. // Close the request body after returning the error to free up resources.
defer r.Body.Close() defer r.Body.Close()
// Don't keep attempting to access this endpoint if the response is a 4XX // Don't keep spamming the endpoint if we've already made too many requests or
// level error which indicates a client mistake. Only retry when the error // if we're not even authenticated correctly. Retrying generally won't fix either
// is due to a server issue (5XX error). // of these issues.
if r.StatusCode >= 400 && r.StatusCode < 500 { if r.StatusCode == http.StatusForbidden ||
r.StatusCode == http.StatusTooManyRequests ||
r.StatusCode == http.StatusUnauthorized {
return backoff.Permanent(r.Error()) return backoff.Permanent(r.Error())
} }
return r.Error() return r.Error()

View File

@@ -6,8 +6,6 @@ import (
"strconv" "strconv"
"sync" "sync"
"github.com/pterodactyl/wings/internal/models"
"emperror.dev/errors" "emperror.dev/errors"
"github.com/apex/log" "github.com/apex/log"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
@@ -180,16 +178,6 @@ func (c *client) SendRestorationStatus(ctx context.Context, backup string, succe
return nil return nil
} }
// SendActivityLogs sends activity logs back to the Panel for processing.
func (c *client) SendActivityLogs(ctx context.Context, activity []models.Activity) error {
resp, err := c.Post(ctx, "/activity", d{"data": activity})
if err != nil {
return errors.WithStackIf(err)
}
_ = resp.Body.Close()
return nil
}
// getServersPaged returns a subset of servers from the Panel API using the // getServersPaged returns a subset of servers from the Panel API using the
// pagination query parameters. // pagination query parameters.
func (c *client) getServersPaged(ctx context.Context, page, limit int) ([]RawServerData, Pagination, error) { func (c *client) getServersPaged(ctx context.Context, page, limit int) ([]RawServerData, Pagination, error) {

View File

@@ -11,11 +11,6 @@ import (
"github.com/pterodactyl/wings/parser" "github.com/pterodactyl/wings/parser"
) )
const (
SftpAuthPassword = SftpAuthRequestType("password")
SftpAuthPublicKey = SftpAuthRequestType("public_key")
)
// A generic type allowing for easy binding use when making requests to API // A generic type allowing for easy binding use when making requests to API
// endpoints that only expect a singular argument or something that would not // endpoints that only expect a singular argument or something that would not
// benefit from being a typed struct. // benefit from being a typed struct.
@@ -68,17 +63,15 @@ type RawServerData struct {
ProcessConfiguration json.RawMessage `json:"process_configuration"` ProcessConfiguration json.RawMessage `json:"process_configuration"`
} }
type SftpAuthRequestType string
// SftpAuthRequest defines the request details that are passed along to the Panel // SftpAuthRequest defines the request details that are passed along to the Panel
// when determining if the credentials provided to Wings are valid. // when determining if the credentials provided to Wings are valid.
type SftpAuthRequest struct { type SftpAuthRequest struct {
Type SftpAuthRequestType `json:"type"` User string `json:"username"`
User string `json:"username"` Pass string `json:"password"`
Pass string `json:"password"` IP string `json:"ip"`
IP string `json:"ip"` SessionID []byte `json:"session_id"`
SessionID []byte `json:"session_id"` ClientVersion []byte `json:"client_version"`
ClientVersion []byte `json:"client_version"` Type string `json:"type"`
} }
// SftpAuthResponse is returned by the Panel when a pair of SFTP credentials // SftpAuthResponse is returned by the Panel when a pair of SFTP credentials
@@ -86,8 +79,8 @@ type SftpAuthRequest struct {
// matched as well as the permissions that are assigned to the authenticated // matched as well as the permissions that are assigned to the authenticated
// user for the SFTP subsystem. // user for the SFTP subsystem.
type SftpAuthResponse struct { type SftpAuthResponse struct {
SSHKeys []string `json:"ssh_keys"`
Server string `json:"server"` Server string `json:"server"`
User string `json:"user"`
Permissions []string `json:"permissions"` Permissions []string `json:"permissions"`
} }
@@ -157,15 +150,9 @@ type BackupRemoteUploadResponse struct {
PartSize int64 `json:"part_size"` PartSize int64 `json:"part_size"`
} }
type BackupPart struct {
ETag string `json:"etag"`
PartNumber int `json:"part_number"`
}
type BackupRequest struct { type BackupRequest struct {
Checksum string `json:"checksum"` Checksum string `json:"checksum"`
ChecksumType string `json:"checksum_type"` ChecksumType string `json:"checksum_type"`
Size int64 `json:"size"` Size int64 `json:"size"`
Successful bool `json:"successful"` Successful bool `json:"successful"`
Parts []BackupPart `json:"parts"`
} }

View File

@@ -4,7 +4,6 @@ import (
"github.com/apex/log" "github.com/apex/log"
"github.com/gin-gonic/gin" "github.com/gin-gonic/gin"
"github.com/pterodactyl/wings/config"
"github.com/pterodactyl/wings/remote" "github.com/pterodactyl/wings/remote"
"github.com/pterodactyl/wings/router/middleware" "github.com/pterodactyl/wings/router/middleware"
wserver "github.com/pterodactyl/wings/server" wserver "github.com/pterodactyl/wings/server"
@@ -16,7 +15,6 @@ func Configure(m *wserver.Manager, client remote.Client) *gin.Engine {
router := gin.New() router := gin.New()
router.Use(gin.Recovery()) router.Use(gin.Recovery())
router.SetTrustedProxies(config.Get().Api.TrustedProxies)
router.Use(middleware.AttachRequestID(), middleware.CaptureErrors(), middleware.SetAccessControlHeaders()) router.Use(middleware.AttachRequestID(), middleware.CaptureErrors(), middleware.SetAccessControlHeaders())
router.Use(middleware.AttachServerManager(m), middleware.AttachApiClient(client)) router.Use(middleware.AttachServerManager(m), middleware.AttachApiClient(client))
// @todo log this into a different file so you can setup IP blocking for abusive requests and such. // @todo log this into a different file so you can setup IP blocking for abusive requests and such.

View File

@@ -180,7 +180,7 @@ func postServerReinstall(c *gin.Context) {
c.Status(http.StatusAccepted) c.Status(http.StatusAccepted)
} }
// Deletes a server from the wings daemon and dissociate its objects. // Deletes a server from the wings daemon and dissociate it's objects.
func deleteServer(c *gin.Context) { func deleteServer(c *gin.Context) {
s := middleware.ExtractServer(c) s := middleware.ExtractServer(c)

View File

@@ -3,7 +3,6 @@ package router
import ( import (
"bufio" "bufio"
"context" "context"
"github.com/pterodactyl/wings/internal/models"
"io" "io"
"mime/multipart" "mime/multipart"
"net/http" "net/http"
@@ -38,15 +37,6 @@ func getServerFileContents(c *gin.Context) {
return return
} }
defer f.Close() defer f.Close()
// Don't allow a named pipe to be opened.
//
// @see https://github.com/pterodactyl/panel/issues/4059
if st.Mode()&os.ModeNamedPipe != 0 {
c.AbortWithStatusJSON(http.StatusBadRequest, gin.H{
"error": "Cannot open files of this type.",
})
return
}
c.Header("X-Mime-Type", st.Mimetype) c.Header("X-Mime-Type", st.Mimetype)
c.Header("Content-Length", strconv.Itoa(int(st.Size()))) c.Header("Content-Length", strconv.Itoa(int(st.Size())))
@@ -132,10 +122,6 @@ func putServerRenameFiles(c *gin.Context) {
// Return nil if the error is an is not exists. // Return nil if the error is an is not exists.
// NOTE: os.IsNotExist() does not work if the error is wrapped. // NOTE: os.IsNotExist() does not work if the error is wrapped.
if errors.Is(err, os.ErrNotExist) { if errors.Is(err, os.ErrNotExist) {
s.Log().WithField("error", err).
WithField("from_path", pf).
WithField("to_path", pt).
Warn("failed to rename: source or target does not exist")
return nil return nil
} }
return err return err
@@ -601,11 +587,6 @@ func postServerUploadFiles(c *gin.Context) {
if err := handleFileUpload(p, s, header); err != nil { if err := handleFileUpload(p, s, header); err != nil {
NewServerError(err, s).Abort(c) NewServerError(err, s).Abort(c)
return return
} else {
s.SaveActivity(s.NewRequestActivity(token.UserUuid, c.ClientIP()), server.ActivityFileUploaded, models.ActivityMeta{
"file": header.Filename,
"directory": filepath.Clean(directory),
})
} }
} }
} }
@@ -623,5 +604,6 @@ func handleFileUpload(p string, s *server.Server, header *multipart.FileHeader)
if err := s.Filesystem().Writefile(p, file); err != nil { if err := s.Filesystem().Writefile(p, file); err != nil {
return err return err
} }
return nil return nil
} }

View File

@@ -32,7 +32,7 @@ func getServerWebsocket(c *gin.Context) {
ctx, cancel := context.WithCancel(c.Request.Context()) ctx, cancel := context.WithCancel(c.Request.Context())
defer cancel() defer cancel()
handler, err := websocket.GetHandler(s, c.Writer, c.Request, c) handler, err := websocket.GetHandler(s, c.Writer, c.Request)
if err != nil { if err != nil {
NewServerError(err, s).Abort(c) NewServerError(err, s).Abort(c)
return return

View File

@@ -12,6 +12,7 @@ import (
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings" "strings"
"sync/atomic"
"time" "time"
"emperror.dev/errors" "emperror.dev/errors"
@@ -29,9 +30,19 @@ import (
"github.com/pterodactyl/wings/router/tokens" "github.com/pterodactyl/wings/router/tokens"
"github.com/pterodactyl/wings/server" "github.com/pterodactyl/wings/server"
"github.com/pterodactyl/wings/server/filesystem" "github.com/pterodactyl/wings/server/filesystem"
"github.com/pterodactyl/wings/system"
) )
const progressWidth = 25 // Number of ticks in the progress bar
const ticks = 25
// 100% / number of ticks = percentage represented by each tick
const tickPercentage = 100 / ticks
type downloadProgress struct {
size int64
progress int64
}
// Data passed over to initiate a server transfer. // Data passed over to initiate a server transfer.
type serverTransferRequest struct { type serverTransferRequest struct {
@@ -84,7 +95,7 @@ func getServerArchive(c *gin.Context) {
return return
} }
// Compute sha256 checksum. // Compute sha1 checksum.
h := sha256.New() h := sha256.New()
f, err := os.Open(archivePath) f, err := os.Open(archivePath)
if err != nil { if err != nil {
@@ -173,35 +184,11 @@ func postServerArchive(c *gin.Context) {
return return
} }
// Get the disk usage of the server (used to calculate the progress of the archive process)
rawSize, err := s.Filesystem().DiskUsage(true)
if err != nil {
sendTransferLog("Failed to get disk usage for server, aborting transfer..")
l.WithField("error", err).Error("failed to get disk usage for server")
return
}
// Create an archive of the entire server's data directory. // Create an archive of the entire server's data directory.
a := &filesystem.Archive{ a := &filesystem.Archive{
BasePath: s.Filesystem().Path(), BasePath: s.Filesystem().Path(),
Progress: filesystem.NewProgress(rawSize),
} }
// Send the archive progress to the websocket every 3 seconds.
ctx, cancel := context.WithCancel(s.Context())
defer cancel()
go func(ctx context.Context, p *filesystem.Progress, t *time.Ticker) {
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
sendTransferLog("Archiving " + p.Progress(progressWidth))
}
}
}(ctx, a.Progress, time.NewTicker(5*time.Second))
// Attempt to get an archive of the server. // Attempt to get an archive of the server.
if err := a.Create(getArchivePath(s.ID())); err != nil { if err := a.Create(getArchivePath(s.ID())); err != nil {
sendTransferLog("An error occurred while archiving the server: " + err.Error()) sendTransferLog("An error occurred while archiving the server: " + err.Error())
@@ -209,12 +196,6 @@ func postServerArchive(c *gin.Context) {
return return
} }
// Cancel the progress ticker.
cancel()
// Show 100% completion.
sendTransferLog("Archiving " + a.Progress.Progress(progressWidth))
sendTransferLog("Successfully created archive, attempting to notify panel..") sendTransferLog("Successfully created archive, attempting to notify panel..")
l.Info("successfully created server transfer archive, notifying panel..") l.Info("successfully created server transfer archive, notifying panel..")
@@ -242,6 +223,12 @@ func postServerArchive(c *gin.Context) {
c.Status(http.StatusAccepted) c.Status(http.StatusAccepted)
} }
func (w *downloadProgress) Write(v []byte) (int, error) {
n := len(v)
atomic.AddInt64(&w.progress, int64(n))
return n, nil
}
// Log helper function to attach all errors and info output to a consistently formatted // Log helper function to attach all errors and info output to a consistently formatted
// log string for easier querying. // log string for easier querying.
func (str serverTransferRequest) log() *log.Entry { func (str serverTransferRequest) log() *log.Entry {
@@ -334,7 +321,7 @@ func postTransfer(c *gin.Context) {
manager := middleware.ExtractManager(c) manager := middleware.ExtractManager(c)
u, err := uuid.Parse(data.ServerID) u, err := uuid.Parse(data.ServerID)
if err != nil { if err != nil {
_ = WithError(c, err) WithError(c, err)
return return
} }
// Force the server ID to be a valid UUID string at this point. If it is not an error // Force the server ID to be a valid UUID string at this point. If it is not an error
@@ -344,12 +331,11 @@ func postTransfer(c *gin.Context) {
data.log().Info("handling incoming server transfer request") data.log().Info("handling incoming server transfer request")
go func(data *serverTransferRequest) { go func(data *serverTransferRequest) {
ctx := context.Background()
hasError := true hasError := true
// Create a new server installer. This will only configure the environment and not // Create a new server installer. This will only configure the environment and not
// run the installer scripts. // run the installer scripts.
i, err := installer.New(ctx, manager, data.Server) i, err := installer.New(context.Background(), manager, data.Server)
if err != nil { if err != nil {
_ = data.sendTransferStatus(manager.Client(), false) _ = data.sendTransferStatus(manager.Client(), false)
data.log().WithField("error", err).Error("failed to validate received server data") data.log().WithField("error", err).Error("failed to validate received server data")
@@ -421,22 +407,25 @@ func postTransfer(c *gin.Context) {
sendTransferLog("Writing archive to disk...") sendTransferLog("Writing archive to disk...")
data.log().Info("writing transfer archive to disk...") data.log().Info("writing transfer archive to disk...")
progress := filesystem.NewProgress(size) // Copy the file.
progress := &downloadProgress{size: size}
// Send the archive progress to the websocket every 3 seconds. ticker := time.NewTicker(3 * time.Second)
ctx, cancel := context.WithCancel(ctx) go func(progress *downloadProgress, t *time.Ticker) {
defer cancel() for range ticker.C {
go func(ctx context.Context, p *filesystem.Progress, t *time.Ticker) { // p = 100 (Downloaded)
defer t.Stop() // size = 1000 (Content-Length)
for { // p / size = 0.1
select { // * 100 = 10% (Multiply by 100 to get a percentage of the download)
case <-ctx.Done(): // 10% / tickPercentage = (10% / (100 / 25)) (Divide by tick percentage to get the number of ticks)
return // 2.5 (Number of ticks as a float64)
case <-t.C: // 2 (convert to an integer)
sendTransferLog("Downloading " + p.Progress(progressWidth)) p := atomic.LoadInt64(&progress.progress)
} // We have to cast these numbers to float in order to get a float result from the division.
width := ((float64(p) / float64(size)) * 100) / tickPercentage
bar := strings.Repeat("=", int(width)) + strings.Repeat(" ", ticks-int(width))
sendTransferLog("Downloading [" + bar + "] " + system.FormatBytes(p) + " / " + system.FormatBytes(progress.size))
} }
}(ctx, progress, time.NewTicker(5*time.Second)) }(progress, ticker)
var reader io.Reader var reader io.Reader
downloadLimit := float64(config.Get().System.Transfers.DownloadLimit) * 1024 * 1024 downloadLimit := float64(config.Get().System.Transfers.DownloadLimit) * 1024 * 1024
@@ -449,16 +438,18 @@ func postTransfer(c *gin.Context) {
buf := make([]byte, 1024*4) buf := make([]byte, 1024*4)
if _, err := io.CopyBuffer(file, io.TeeReader(reader, progress), buf); err != nil { if _, err := io.CopyBuffer(file, io.TeeReader(reader, progress), buf); err != nil {
ticker.Stop()
_ = file.Close() _ = file.Close()
sendTransferLog("Failed while writing archive file to disk: " + err.Error()) sendTransferLog("Failed while writing archive file to disk: " + err.Error())
data.log().WithField("error", err).Error("failed to copy archive file to disk") data.log().WithField("error", err).Error("failed to copy archive file to disk")
return return
} }
cancel() ticker.Stop()
// Show 100% completion. // Show 100% completion.
sendTransferLog("Downloading " + progress.Progress(progressWidth)) humanSize := system.FormatBytes(progress.size)
sendTransferLog("Downloading [" + strings.Repeat("=", ticks) + "] " + humanSize + " / " + humanSize)
if err := file.Close(); err != nil { if err := file.Close(); err != nil {
data.log().WithField("error", err).Error("unable to close archive file on local filesystem") data.log().WithField("error", err).Error("unable to close archive file on local filesystem")

View File

@@ -8,7 +8,6 @@ type UploadPayload struct {
jwt.Payload jwt.Payload
ServerUuid string `json:"server_uuid"` ServerUuid string `json:"server_uuid"`
UserUuid string `json:"user_uuid"`
UniqueId string `json:"unique_id"` UniqueId string `json:"unique_id"`
} }

View File

@@ -7,6 +7,7 @@ import (
"github.com/apex/log" "github.com/apex/log"
"github.com/gbrlsnchs/jwt/v3" "github.com/gbrlsnchs/jwt/v3"
"github.com/goccy/go-json"
) )
// The time at which Wings was booted. No JWT's created before this time are allowed to // The time at which Wings was booted. No JWT's created before this time are allowed to
@@ -34,15 +35,15 @@ func DenyJTI(jti string) {
denylist.Store(jti, time.Now()) denylist.Store(jti, time.Now())
} }
// WebsocketPayload defines the JWT payload for a websocket connection. This JWT is passed along to // A JWT payload for Websocket connections. This JWT is passed along to the Websocket after
// the websocket after it has been connected to by sending an "auth" event. // it has been connected to by sending an "auth" event.
type WebsocketPayload struct { type WebsocketPayload struct {
jwt.Payload jwt.Payload
sync.RWMutex sync.RWMutex
UserUUID string `json:"user_uuid"` UserID json.Number `json:"user_id"`
ServerUUID string `json:"server_uuid"` ServerUUID string `json:"server_uuid"`
Permissions []string `json:"permissions"` Permissions []string `json:"permissions"`
} }
// Returns the JWT payload. // Returns the JWT payload.

View File

@@ -164,6 +164,5 @@ func (h *Handler) listenForServerEvents(ctx context.Context) error {
if err != nil { if err != nil {
return errors.WithStack(err) return errors.WithStack(err)
} }
return nil return nil
} }

View File

@@ -3,7 +3,6 @@ package websocket
import ( import (
"context" "context"
"fmt" "fmt"
"github.com/pterodactyl/wings/internal/models"
"net/http" "net/http"
"strings" "strings"
"sync" "sync"
@@ -12,7 +11,6 @@ import (
"emperror.dev/errors" "emperror.dev/errors"
"github.com/apex/log" "github.com/apex/log"
"github.com/gbrlsnchs/jwt/v3" "github.com/gbrlsnchs/jwt/v3"
"github.com/gin-gonic/gin"
"github.com/goccy/go-json" "github.com/goccy/go-json"
"github.com/google/uuid" "github.com/google/uuid"
"github.com/gorilla/websocket" "github.com/gorilla/websocket"
@@ -42,7 +40,6 @@ type Handler struct {
Connection *websocket.Conn `json:"-"` Connection *websocket.Conn `json:"-"`
jwt *tokens.WebsocketPayload jwt *tokens.WebsocketPayload
server *server.Server server *server.Server
ra server.RequestActivity
uuid uuid.UUID uuid uuid.UUID
} }
@@ -80,7 +77,7 @@ func NewTokenPayload(token []byte) (*tokens.WebsocketPayload, error) {
} }
// GetHandler returns a new websocket handler using the context provided. // GetHandler returns a new websocket handler using the context provided.
func GetHandler(s *server.Server, w http.ResponseWriter, r *http.Request, c *gin.Context) (*Handler, error) { func GetHandler(s *server.Server, w http.ResponseWriter, r *http.Request) (*Handler, error) {
upgrader := websocket.Upgrader{ upgrader := websocket.Upgrader{
// Ensure that the websocket request is originating from the Panel itself, // Ensure that the websocket request is originating from the Panel itself,
// and not some other location. // and not some other location.
@@ -112,7 +109,6 @@ func GetHandler(s *server.Server, w http.ResponseWriter, r *http.Request, c *gin
Connection: conn, Connection: conn,
jwt: nil, jwt: nil,
server: s, server: s,
ra: s.NewRequestActivity("", c.ClientIP()),
uuid: u, uuid: u,
}, nil }, nil
} }
@@ -268,7 +264,6 @@ func (h *Handler) GetJwt() *tokens.WebsocketPayload {
// setJwt sets the JWT for the websocket in a race-safe manner. // setJwt sets the JWT for the websocket in a race-safe manner.
func (h *Handler) setJwt(token *tokens.WebsocketPayload) { func (h *Handler) setJwt(token *tokens.WebsocketPayload) {
h.Lock() h.Lock()
h.ra = h.ra.SetUser(token.UserUUID)
h.jwt = token h.jwt = token
h.Unlock() h.Unlock()
} }
@@ -370,10 +365,6 @@ func (h *Handler) HandleInbound(ctx context.Context, m Message) error {
return nil return nil
} }
if err == nil {
h.server.SaveActivity(h.ra, models.Event(server.ActivityPowerPrefix+action), nil)
}
return err return err
} }
case SendServerLogsEvent: case SendServerLogsEvent:
@@ -430,13 +421,7 @@ func (h *Handler) HandleInbound(ctx context.Context, m Message) error {
} }
} }
if err := h.server.Environment.SendCommand(strings.Join(m.Args, "")); err != nil { return h.server.Environment.SendCommand(strings.Join(m.Args, ""))
return err
}
h.server.SaveActivity(h.ra, server.ActivityConsoleCommand, models.ActivityMeta{
"command": strings.Join(m.Args, ""),
})
return nil
} }
} }

View File

@@ -1,5 +1,5 @@
Name: ptero-wings Name: ptero-wings
Version: 1.7.0 Version: 1.5.3
Release: 1%{?dist} Release: 1%{?dist}
Summary: The server control plane for Pterodactyl Panel. Written from the ground-up with security, speed, and stability in mind. Summary: The server control plane for Pterodactyl Panel. Written from the ground-up with security, speed, and stability in mind.
BuildArch: x86_64 BuildArch: x86_64
@@ -91,9 +91,6 @@ rm -rf /var/log/pterodactyl
wings --version wings --version
%changelog %changelog
* Wed Sep 14 2022 Chance Callahan <ccallaha@redhat.com> - 1.7.0-1
- Updating specfile to match stable release.
* Wed Oct 27 2021 Capitol Hosting Solutions Systems Engineering <syseng@chs.gg> - 1.5.3-1 * Wed Oct 27 2021 Capitol Hosting Solutions Systems Engineering <syseng@chs.gg> - 1.5.3-1
- specfile by Capitol Hosting Solutions, Upstream by Pterodactyl - specfile by Capitol Hosting Solutions, Upstream by Pterodactyl
- Rebased for https://github.com/pterodactyl/wings/releases/tag/v1.5.3 - Rebased for https://github.com/pterodactyl/wings/releases/tag/v1.5.3

View File

@@ -1,64 +0,0 @@
package server
import (
"context"
"emperror.dev/errors"
"github.com/pterodactyl/wings/internal/database"
"github.com/pterodactyl/wings/internal/models"
"time"
)
const ActivityPowerPrefix = "server:power."
const (
ActivityConsoleCommand = models.Event("server:console.command")
ActivitySftpWrite = models.Event("server:sftp.write")
ActivitySftpCreate = models.Event("server:sftp.create")
ActivitySftpCreateDirectory = models.Event("server:sftp.create-directory")
ActivitySftpRename = models.Event("server:sftp.rename")
ActivitySftpDelete = models.Event("server:sftp.delete")
ActivityFileUploaded = models.Event("server:file.uploaded")
)
// RequestActivity is a wrapper around a LoggedEvent that is able to track additional request
// specific metadata including the specific user and IP address associated with all subsequent
// events. The internal logged event structure can be extracted by calling RequestEvent.Event().
type RequestActivity struct {
server string
user string
ip string
}
// Event returns the underlying logged event from the RequestEvent instance and sets the
// specific event and metadata on it.
func (ra RequestActivity) Event(event models.Event, metadata models.ActivityMeta) *models.Activity {
a := models.Activity{Server: ra.server, IP: ra.ip, Event: event, Metadata: metadata}
return a.SetUser(ra.user)
}
// SetUser clones the RequestActivity struct and sets a new user value on the copy
// before returning it.
func (ra RequestActivity) SetUser(u string) RequestActivity {
c := ra
c.user = u
return c
}
func (s *Server) NewRequestActivity(user string, ip string) RequestActivity {
return RequestActivity{server: s.ID(), user: user, ip: ip}
}
// SaveActivity saves an activity entry to the database in a background routine. If an error is
// encountered it is logged but not returned to the caller.
func (s *Server) SaveActivity(a RequestActivity, event models.Event, metadata models.ActivityMeta) {
ctx, cancel := context.WithTimeout(s.Context(), time.Second*3)
go func() {
defer cancel()
if tx := database.Instance().WithContext(ctx).Create(a.Event(event, metadata)); tx.Error != nil {
s.Log().WithField("error", errors.WithStack(tx.Error)).
WithField("event", event).
Error("activity: failed to save event")
}
}()
}

View File

@@ -32,7 +32,7 @@ type RestoreCallback func(file string, r io.Reader, mode fs.FileMode, atime, mti
// noinspection GoNameStartsWithPackageName // noinspection GoNameStartsWithPackageName
type BackupInterface interface { type BackupInterface interface {
// SetClient sets the API request client on the backup interface. // SetClient sets the API request client on the backup interface.
SetClient(remote.Client) SetClient(c remote.Client)
// Identifier returns the UUID of this backup as tracked by the panel // Identifier returns the UUID of this backup as tracked by the panel
// instance. // instance.
Identifier() string Identifier() string
@@ -41,7 +41,7 @@ type BackupInterface interface {
WithLogContext(map[string]interface{}) WithLogContext(map[string]interface{})
// Generate creates a backup in whatever the configured source for the // Generate creates a backup in whatever the configured source for the
// specific implementation is. // specific implementation is.
Generate(context.Context, string, string) (*ArchiveDetails, error) Generate(ctx context.Context, basePath string, ignore string) (*ArchiveDetails, error)
// Ignored returns the ignored files for this backup instance. // Ignored returns the ignored files for this backup instance.
Ignored() string Ignored() string
// Checksum returns a SHA1 checksum for the generated backup. // Checksum returns a SHA1 checksum for the generated backup.
@@ -53,13 +53,13 @@ type BackupInterface interface {
// to store it until it is moved to the final spot. // to store it until it is moved to the final spot.
Path() string Path() string
// Details returns details about the archive. // Details returns details about the archive.
Details(context.Context, []remote.BackupPart) (*ArchiveDetails, error) Details(ctx context.Context) (*ArchiveDetails, error)
// Remove removes a backup file. // Remove removes a backup file.
Remove() error Remove() error
// Restore is called when a backup is ready to be restored to the disk from // Restore is called when a backup is ready to be restored to the disk from
// the given source. Not every backup implementation will support this nor // the given source. Not every backup implementation will support this nor
// will every implementation require a reader be provided. // will every implementation require a reader be provided.
Restore(context.Context, io.Reader, RestoreCallback) error Restore(ctx context.Context, reader io.Reader, callback RestoreCallback) error
} }
type Backup struct { type Backup struct {
@@ -119,8 +119,8 @@ func (b *Backup) Checksum() ([]byte, error) {
// Details returns both the checksum and size of the archive currently stored on // Details returns both the checksum and size of the archive currently stored on
// the disk to the caller. // the disk to the caller.
func (b *Backup) Details(ctx context.Context, parts []remote.BackupPart) (*ArchiveDetails, error) { func (b *Backup) Details(ctx context.Context) (*ArchiveDetails, error) {
ad := ArchiveDetails{ChecksumType: "sha1", Parts: parts} ad := ArchiveDetails{ChecksumType: "sha1"}
g, ctx := errgroup.WithContext(ctx) g, ctx := errgroup.WithContext(ctx)
g.Go(func() error { g.Go(func() error {
@@ -162,10 +162,9 @@ func (b *Backup) log() *log.Entry {
} }
type ArchiveDetails struct { type ArchiveDetails struct {
Checksum string `json:"checksum"` Checksum string `json:"checksum"`
ChecksumType string `json:"checksum_type"` ChecksumType string `json:"checksum_type"`
Size int64 `json:"size"` Size int64 `json:"size"`
Parts []remote.BackupPart `json:"parts"`
} }
// ToRequest returns a request object. // ToRequest returns a request object.
@@ -175,6 +174,5 @@ func (ad *ArchiveDetails) ToRequest(successful bool) remote.BackupRequest {
ChecksumType: ad.ChecksumType, ChecksumType: ad.ChecksumType,
Size: ad.Size, Size: ad.Size,
Successful: successful, Successful: successful,
Parts: ad.Parts,
} }
} }

View File

@@ -69,7 +69,7 @@ func (b *LocalBackup) Generate(ctx context.Context, basePath, ignore string) (*A
} }
b.log().Info("created backup successfully") b.log().Info("created backup successfully")
ad, err := b.Details(ctx, nil) ad, err := b.Details(ctx)
if err != nil { if err != nil {
return nil, errors.WrapIf(err, "backup: failed to get archive details for local backup") return nil, errors.WrapIf(err, "backup: failed to get archive details for local backup")
} }

View File

@@ -71,11 +71,10 @@ func (s *S3Backup) Generate(ctx context.Context, basePath, ignore string) (*Arch
} }
defer rc.Close() defer rc.Close()
parts, err := s.generateRemoteRequest(ctx, rc) if err := s.generateRemoteRequest(ctx, rc); err != nil {
if err != nil {
return nil, err return nil, err
} }
ad, err := s.Details(ctx, parts) ad, err := s.Details(ctx)
if err != nil { if err != nil {
return nil, errors.WrapIf(err, "backup: failed to get archive details after upload") return nil, errors.WrapIf(err, "backup: failed to get archive details after upload")
} }
@@ -126,20 +125,20 @@ func (s *S3Backup) Restore(ctx context.Context, r io.Reader, callback RestoreCal
} }
// Generates the remote S3 request and begins the upload. // Generates the remote S3 request and begins the upload.
func (s *S3Backup) generateRemoteRequest(ctx context.Context, rc io.ReadCloser) ([]remote.BackupPart, error) { func (s *S3Backup) generateRemoteRequest(ctx context.Context, rc io.ReadCloser) error {
defer rc.Close() defer rc.Close()
s.log().Debug("attempting to get size of backup...") s.log().Debug("attempting to get size of backup...")
size, err := s.Backup.Size() size, err := s.Backup.Size()
if err != nil { if err != nil {
return nil, err return err
} }
s.log().WithField("size", size).Debug("got size of backup") s.log().WithField("size", size).Debug("got size of backup")
s.log().Debug("attempting to get S3 upload urls from Panel...") s.log().Debug("attempting to get S3 upload urls from Panel...")
urls, err := s.client.GetBackupRemoteUploadURLs(context.Background(), s.Backup.Uuid, size) urls, err := s.client.GetBackupRemoteUploadURLs(context.Background(), s.Backup.Uuid, size)
if err != nil { if err != nil {
return nil, err return err
} }
s.log().Debug("got S3 upload urls from the Panel") s.log().Debug("got S3 upload urls from the Panel")
s.log().WithField("parts", len(urls.Parts)).Info("attempting to upload backup to s3 endpoint...") s.log().WithField("parts", len(urls.Parts)).Info("attempting to upload backup to s3 endpoint...")
@@ -157,26 +156,22 @@ func (s *S3Backup) generateRemoteRequest(ctx context.Context, rc io.ReadCloser)
} }
// Attempt to upload the part. // Attempt to upload the part.
etag, err := uploader.uploadPart(ctx, part, partSize) if _, err := uploader.uploadPart(ctx, part, partSize); err != nil {
if err != nil {
s.log().WithField("part_id", i+1).WithError(err).Warn("failed to upload part") s.log().WithField("part_id", i+1).WithError(err).Warn("failed to upload part")
return nil, err return err
} }
uploader.uploadedParts = append(uploader.uploadedParts, remote.BackupPart{
ETag: etag,
PartNumber: i + 1,
})
s.log().WithField("part_id", i+1).Info("successfully uploaded backup part") s.log().WithField("part_id", i+1).Info("successfully uploaded backup part")
} }
s.log().WithField("parts", len(urls.Parts)).Info("backup has been successfully uploaded") s.log().WithField("parts", len(urls.Parts)).Info("backup has been successfully uploaded")
return uploader.uploadedParts, nil return nil
} }
type s3FileUploader struct { type s3FileUploader struct {
io.ReadCloser io.ReadCloser
client *http.Client client *http.Client
uploadedParts []remote.BackupPart
} }
// newS3FileUploader returns a new file uploader instance. // newS3FileUploader returns a new file uploader instance.

View File

@@ -16,11 +16,6 @@ type EggConfiguration struct {
FileDenylist []string `json:"file_denylist"` FileDenylist []string `json:"file_denylist"`
} }
type ConfigurationMeta struct {
Name string `json:"name"`
Description string `json:"description"`
}
type Configuration struct { type Configuration struct {
mu sync.RWMutex mu sync.RWMutex
@@ -29,8 +24,6 @@ type Configuration struct {
// docker containers as well as in log output. // docker containers as well as in log output.
Uuid string `json:"uuid"` Uuid string `json:"uuid"`
Meta ConfigurationMeta `json:"meta"`
// Whether or not the server is in a suspended state. Suspended servers cannot // Whether or not the server is in a suspended state. Suspended servers cannot
// be started or modified except in certain scenarios by an admin user. // be started or modified except in certain scenarios by an admin user.
Suspended bool `json:"suspended"` Suspended bool `json:"suspended"`

View File

@@ -8,7 +8,6 @@ import (
"path/filepath" "path/filepath"
"strings" "strings"
"sync" "sync"
"sync/atomic"
"emperror.dev/errors" "emperror.dev/errors"
"github.com/apex/log" "github.com/apex/log"
@@ -18,7 +17,6 @@ import (
ignore "github.com/sabhiram/go-gitignore" ignore "github.com/sabhiram/go-gitignore"
"github.com/pterodactyl/wings/config" "github.com/pterodactyl/wings/config"
"github.com/pterodactyl/wings/system"
) )
const memory = 4 * 1024 const memory = 4 * 1024
@@ -30,62 +28,6 @@ var pool = sync.Pool{
}, },
} }
// Progress is used to track the progress of any I/O operation that are being
// performed.
type Progress struct {
// written is the total size of the files that have been written to the writer.
written int64
// Total is the total size of the archive in bytes.
total int64
// w .
w io.Writer
}
// NewProgress .
func NewProgress(total int64) *Progress {
return &Progress{total: total}
}
// Written returns the total number of bytes written.
// This function should be used when the progress is tracking data being written.
func (p *Progress) Written() int64 {
return atomic.LoadInt64(&p.written)
}
// Total returns the total size in bytes.
func (p *Progress) Total() int64 {
return atomic.LoadInt64(&p.total)
}
// Write totals the number of bytes that have been written to the writer.
func (p *Progress) Write(v []byte) (int, error) {
n := len(v)
atomic.AddInt64(&p.written, int64(n))
if p.w != nil {
return p.w.Write(v)
}
return n, nil
}
// Progress returns a formatted progress string for the current progress.
func (p *Progress) Progress(width int) string {
current := p.Written()
total := p.Total()
// v = 100 (Progress)
// size = 1000 (Content-Length)
// p / size = 0.1
// * 100 = 10% (Multiply by 100 to get a percentage of the download)
// 10% / tickPercentage = (10% / (100 / 25)) (Divide by tick percentage to get the number of ticks)
// 2.5 (Number of ticks as a float64)
// 2 (convert to an integer)
// We have to cast these numbers to float in order to get a float result from the division.
ticks := ((float64(current) / float64(total)) * 100) / (float64(100) / float64(width))
bar := strings.Repeat("=", int(ticks)) + strings.Repeat(" ", width-int(ticks))
return "[" + bar + "] " + system.FormatBytes(current) + " / " + system.FormatBytes(total)
}
type Archive struct { type Archive struct {
// BasePath is the absolute path to create the archive from where Files and Ignore are // BasePath is the absolute path to create the archive from where Files and Ignore are
// relative to. // relative to.
@@ -98,13 +40,10 @@ type Archive struct {
// Files specifies the files to archive, this takes priority over the Ignore option, if // Files specifies the files to archive, this takes priority over the Ignore option, if
// unspecified, all files in the BasePath will be archived unless Ignore is set. // unspecified, all files in the BasePath will be archived unless Ignore is set.
Files []string Files []string
// Progress wraps the writer of the archive to pass through the progress tracker.
Progress *Progress
} }
// Create creates an archive at dst with all the files defined in the // Create creates an archive at dst with all of the files defined in the
// included Files array. // included files struct.
func (a *Archive) Create(dst string) error { func (a *Archive) Create(dst string) error {
f, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600) f, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil { if err != nil {
@@ -123,34 +62,13 @@ func (a *Archive) Create(dst string) error {
writer = f writer = f
} }
// Choose which compression level to use based on the compression_level configuration option
var compressionLevel int
switch config.Get().System.Backups.CompressionLevel {
case "none":
compressionLevel = pgzip.NoCompression
case "best_compression":
compressionLevel = pgzip.BestCompression
case "best_speed":
fallthrough
default:
compressionLevel = pgzip.BestSpeed
}
// Create a new gzip writer around the file. // Create a new gzip writer around the file.
gw, _ := pgzip.NewWriterLevel(writer, compressionLevel) gw, _ := pgzip.NewWriterLevel(writer, pgzip.BestSpeed)
_ = gw.SetConcurrency(1<<20, 1) _ = gw.SetConcurrency(1<<20, 1)
defer gw.Close() defer gw.Close()
var pw io.Writer
if a.Progress != nil {
a.Progress.w = gw
pw = a.Progress
} else {
pw = gw
}
// Create a new tar writer around the gzip writer. // Create a new tar writer around the gzip writer.
tw := tar.NewWriter(pw) tw := tar.NewWriter(gw)
defer tw.Close() defer tw.Close()
// Configure godirwalk. // Configure godirwalk.
@@ -185,7 +103,7 @@ func (a *Archive) Create(dst string) error {
// being generated. // being generated.
func (a *Archive) callback(tw *tar.Writer, opts ...func(path string, relative string) error) func(path string, de *godirwalk.Dirent) error { func (a *Archive) callback(tw *tar.Writer, opts ...func(path string, relative string) error) func(path string, de *godirwalk.Dirent) error {
return func(path string, de *godirwalk.Dirent) error { return func(path string, de *godirwalk.Dirent) error {
// Skip directories because we are walking them recursively. // Skip directories because we walking them recursively.
if de.IsDir() { if de.IsDir() {
return nil return nil
} }
@@ -212,7 +130,7 @@ func (a *Archive) withFilesCallback(tw *tar.Writer) func(path string, de *godirw
for _, f := range a.Files { for _, f := range a.Files {
// If the given doesn't match, or doesn't have the same prefix continue // If the given doesn't match, or doesn't have the same prefix continue
// to the next item in the loop. // to the next item in the loop.
if p != f && !strings.HasPrefix(strings.TrimSuffix(p, "/")+"/", f) { if p != f && !strings.HasPrefix(p, f) {
continue continue
} }
@@ -230,7 +148,7 @@ func (a *Archive) withFilesCallback(tw *tar.Writer) func(path string, de *godirw
// Adds a given file path to the final archive being created. // Adds a given file path to the final archive being created.
func (a *Archive) addToArchive(p string, rp string, w *tar.Writer) error { func (a *Archive) addToArchive(p string, rp string, w *tar.Writer) error {
// Lstat the file, this will give us the same information as Stat except that it will not // Lstat the file, this will give us the same information as Stat except that it will not
// follow a symlink to its target automatically. This is important to avoid including // follow a symlink to it's target automatically. This is important to avoid including
// files that exist outside the server root unintentionally in the backup. // files that exist outside the server root unintentionally in the backup.
s, err := os.Lstat(p) s, err := os.Lstat(p)
if err != nil { if err != nil {

View File

@@ -5,12 +5,9 @@ import (
"archive/zip" "archive/zip"
"compress/gzip" "compress/gzip"
"fmt" "fmt"
gzip2 "github.com/klauspost/compress/gzip"
zip2 "github.com/klauspost/compress/zip"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
"reflect"
"strings" "strings"
"sync/atomic" "sync/atomic"
"time" "time"
@@ -175,26 +172,13 @@ func ExtractNameFromArchive(f archiver.File) string {
return f.Name() return f.Name()
} }
switch s := sys.(type) { switch s := sys.(type) {
case *zip.FileHeader:
return s.Name
case *zip2.FileHeader:
return s.Name
case *tar.Header: case *tar.Header:
return s.Name return s.Name
case *gzip.Header: case *gzip.Header:
return s.Name return s.Name
case *gzip2.Header: case *zip.FileHeader:
return s.Name return s.Name
default: default:
// At this point we cannot figure out what type of archive this might be so
// just try to find the name field in the struct. If it is found return it.
field := reflect.Indirect(reflect.ValueOf(sys)).FieldByName("Name")
if field.IsValid() {
return field.String()
}
// Fallback to the basename of the file at this point. There is nothing we can really
// do to try and figure out what the underlying directory of the file is supposed to
// be since it didn't implement a name field.
return f.Name() return f.Name()
} }
} }

View File

@@ -71,7 +71,7 @@ func (fs *Filesystem) HasSpaceAvailable(allowStaleValue bool) bool {
// If space is -1 or 0 just return true, means they're allowed unlimited. // If space is -1 or 0 just return true, means they're allowed unlimited.
// //
// Technically we could skip disk space calculation because we don't need to check if the // Technically we could skip disk space calculation because we don't need to check if the
// server exceeds its limit but because this method caches the disk usage it would be best // server exceeds it's limit but because this method caches the disk usage it would be best
// to calculate the disk usage and always return true. // to calculate the disk usage and always return true.
if fs.MaxDisk() == 0 { if fs.MaxDisk() == 0 {
return true return true

View File

@@ -115,6 +115,19 @@ func (fs *Filesystem) Touch(p string, flag int) (*os.File, error) {
return f, nil return f, nil
} }
// Reads a file on the system and returns it as a byte representation in a file
// reader. This is not the most memory efficient usage since it will be reading the
// entirety of the file into memory.
func (fs *Filesystem) Readfile(p string, w io.Writer) error {
file, _, err := fs.File(p)
if err != nil {
return err
}
defer file.Close()
_, err = bufio.NewReader(file).WriteTo(w)
return err
}
// Writefile writes a file to the system. If the file does not already exist one // Writefile writes a file to the system. If the file does not already exist one
// will be created. This will also properly recalculate the disk space used by // will be created. This will also properly recalculate the disk space used by
// the server when writing new files or modifying existing ones. // the server when writing new files or modifying existing ones.
@@ -159,7 +172,7 @@ func (fs *Filesystem) Writefile(p string, r io.Reader) error {
// Adjust the disk usage to account for the old size and the new size of the file. // Adjust the disk usage to account for the old size and the new size of the file.
fs.addDisk(sz - currentSize) fs.addDisk(sz - currentSize)
return fs.unsafeChown(cleaned) return fs.Chown(cleaned)
} }
// Creates a new directory (name) at a specified path (p) for the server. // Creates a new directory (name) at a specified path (p) for the server.
@@ -171,16 +184,16 @@ func (fs *Filesystem) CreateDirectory(name string, p string) error {
return os.MkdirAll(cleaned, 0o755) return os.MkdirAll(cleaned, 0o755)
} }
// Rename moves (or renames) a file or directory. // Moves (or renames) a file or directory.
func (fs *Filesystem) Rename(from string, to string) error { func (fs *Filesystem) Rename(from string, to string) error {
cleanedFrom, err := fs.SafePath(from) cleanedFrom, err := fs.SafePath(from)
if err != nil { if err != nil {
return errors.WithStack(err) return err
} }
cleanedTo, err := fs.SafePath(to) cleanedTo, err := fs.SafePath(to)
if err != nil { if err != nil {
return errors.WithStack(err) return err
} }
// If the target file or directory already exists the rename function will fail, so just // If the target file or directory already exists the rename function will fail, so just
@@ -202,10 +215,7 @@ func (fs *Filesystem) Rename(from string, to string) error {
} }
} }
if err := os.Rename(cleanedFrom, cleanedTo); err != nil { return os.Rename(cleanedFrom, cleanedTo)
return errors.WithStack(err)
}
return nil
} }
// Recursively iterates over a file or directory and sets the permissions on all of the // Recursively iterates over a file or directory and sets the permissions on all of the
@@ -217,12 +227,7 @@ func (fs *Filesystem) Chown(path string) error {
if err != nil { if err != nil {
return err return err
} }
return fs.unsafeChown(cleaned)
}
// unsafeChown chowns the given path, without checking if the path is safe. This should only be used
// when the path has already been checked.
func (fs *Filesystem) unsafeChown(path string) error {
if fs.isTest { if fs.isTest {
return nil return nil
} }
@@ -231,19 +236,19 @@ func (fs *Filesystem) unsafeChown(path string) error {
gid := config.Get().System.User.Gid gid := config.Get().System.User.Gid
// Start by just chowning the initial path that we received. // Start by just chowning the initial path that we received.
if err := os.Chown(path, uid, gid); err != nil { if err := os.Chown(cleaned, uid, gid); err != nil {
return errors.Wrap(err, "server/filesystem: chown: failed to chown path") return errors.Wrap(err, "server/filesystem: chown: failed to chown path")
} }
// If this is not a directory we can now return from the function, there is nothing // If this is not a directory we can now return from the function, there is nothing
// left that we need to do. // left that we need to do.
if st, err := os.Stat(path); err != nil || !st.IsDir() { if st, err := os.Stat(cleaned); err != nil || !st.IsDir() {
return nil return nil
} }
// If this was a directory, begin walking over its contents recursively and ensure that all // If this was a directory, begin walking over its contents recursively and ensure that all
// of the subfiles and directories get their permissions updated as well. // of the subfiles and directories get their permissions updated as well.
err := godirwalk.Walk(path, &godirwalk.Options{ err = godirwalk.Walk(cleaned, &godirwalk.Options{
Unsorted: true, Unsorted: true,
Callback: func(p string, e *godirwalk.Dirent) error { Callback: func(p string, e *godirwalk.Dirent) error {
// Do not attempt to chown a symlink. Go's os.Chown function will affect the symlink // Do not attempt to chown a symlink. Go's os.Chown function will affect the symlink
@@ -260,6 +265,7 @@ func (fs *Filesystem) unsafeChown(path string) error {
return os.Chown(p, uid, gid) return os.Chown(p, uid, gid)
}, },
}) })
return errors.Wrap(err, "server/filesystem: chown: failed to chown during walk function") return errors.Wrap(err, "server/filesystem: chown: failed to chown during walk function")
} }
@@ -381,9 +387,10 @@ func (fs *Filesystem) TruncateRootDirectory() error {
// Delete removes a file or folder from the system. Prevents the user from // Delete removes a file or folder from the system. Prevents the user from
// accidentally (or maliciously) removing their root server data directory. // accidentally (or maliciously) removing their root server data directory.
func (fs *Filesystem) Delete(p string) error { func (fs *Filesystem) Delete(p string) error {
wg := sync.WaitGroup{}
// This is one of the few (only?) places in the codebase where we're explicitly not using // This is one of the few (only?) places in the codebase where we're explicitly not using
// the SafePath functionality when working with user provided input. If we did, you would // the SafePath functionality when working with user provided input. If we did, you would
// not be able to delete a file that is a symlink pointing to a location outside the data // not be able to delete a file that is a symlink pointing to a location outside of the data
// directory. // directory.
// //
// We also want to avoid resolving a symlink that points _within_ the data directory and thus // We also want to avoid resolving a symlink that points _within_ the data directory and thus
@@ -400,66 +407,26 @@ func (fs *Filesystem) Delete(p string) error {
return errors.New("cannot delete root server directory") return errors.New("cannot delete root server directory")
} }
st, err := os.Lstat(resolved) if st, err := os.Lstat(resolved); err != nil {
if err != nil {
if !os.IsNotExist(err) { if !os.IsNotExist(err) {
fs.error(err).Warn("error while attempting to stat file before deletion") fs.error(err).Warn("error while attempting to stat file before deletion")
return err
}
// The following logic is used to handle a case where a user attempts to
// delete a file that does not exist through a directory symlink.
// We don't want to reveal that the file does not exist, so we validate
// the path of the symlink and return a bad path error if it is invalid.
// The requested file or directory doesn't exist, so at this point we
// need to iterate up the path chain until we hit a directory that
// _does_ exist and can be validated.
parts := strings.Split(filepath.Dir(resolved), "/")
// Range over all the path parts and form directory paths from the end
// moving up until we have a valid resolution, or we run out of paths to
// try.
for k := range parts {
try := strings.Join(parts[:(len(parts)-k)], "/")
if !fs.unsafeIsInDataDirectory(try) {
break
}
t, err := filepath.EvalSymlinks(try)
if err == nil {
if !fs.unsafeIsInDataDirectory(t) {
return NewBadPathResolution(p, t)
}
break
}
}
// Always return early if the file does not exist.
return nil
}
// If the file is not a symlink, we need to check that it is not within a
// symlinked directory that points outside the data directory.
if st.Mode()&os.ModeSymlink == 0 {
ep, err := filepath.EvalSymlinks(resolved)
if err != nil {
if !os.IsNotExist(err) {
return err
}
} else if !fs.unsafeIsInDataDirectory(ep) {
return NewBadPathResolution(p, ep)
}
}
if st.IsDir() {
if s, err := fs.DirectorySize(resolved); err == nil {
fs.addDisk(-s)
} }
} else { } else {
fs.addDisk(-st.Size()) if !st.IsDir() {
fs.addDisk(-st.Size())
} else {
wg.Add(1)
go func(wg *sync.WaitGroup, st os.FileInfo, resolved string) {
defer wg.Done()
if s, err := fs.DirectorySize(resolved); err == nil {
fs.addDisk(-s)
}
}(&wg, st, resolved)
}
} }
wg.Wait()
return os.RemoveAll(resolved) return os.RemoveAll(resolved)
} }
@@ -525,11 +492,7 @@ func (fs *Filesystem) ListDirectory(p string) ([]Stat, error) {
cleanedp, _ = fs.SafePath(filepath.Join(cleaned, f.Name())) cleanedp, _ = fs.SafePath(filepath.Join(cleaned, f.Name()))
} }
// Don't try to detect the type on a pipe — this will just hang the application and if cleanedp != "" {
// you'll never get a response back.
//
// @see https://github.com/pterodactyl/panel/issues/4059
if cleanedp != "" && f.Mode()&os.ModeNamedPipe == 0 {
m, _ = mimetype.DetectFile(filepath.Join(cleaned, f.Name())) m, _ = mimetype.DetectFile(filepath.Join(cleaned, f.Name()))
} else { } else {
// Just pass this for an unknown type because the file could not safely be resolved within // Just pass this for an unknown type because the file could not safely be resolved within

View File

@@ -1,7 +1,6 @@
package filesystem package filesystem
import ( import (
"bufio"
"bytes" "bytes"
"errors" "errors"
"math/rand" "math/rand"
@@ -45,14 +44,6 @@ type rootFs struct {
root string root string
} }
func getFileContent(file *os.File) string {
var w bytes.Buffer
if _, err := bufio.NewReader(file).WriteTo(&w); err != nil {
panic(err)
}
return w.String()
}
func (rfs *rootFs) CreateServerFile(p string, c []byte) error { func (rfs *rootFs) CreateServerFile(p string, c []byte) error {
f, err := os.Create(filepath.Join(rfs.root, "/server", p)) f, err := os.Create(filepath.Join(rfs.root, "/server", p))
@@ -84,6 +75,54 @@ func (rfs *rootFs) reset() {
} }
} }
func TestFilesystem_Readfile(t *testing.T) {
g := Goblin(t)
fs, rfs := NewFs()
g.Describe("Readfile", func() {
buf := &bytes.Buffer{}
g.It("opens a file if it exists on the system", func() {
err := rfs.CreateServerFileFromString("test.txt", "testing")
g.Assert(err).IsNil()
err = fs.Readfile("test.txt", buf)
g.Assert(err).IsNil()
g.Assert(buf.String()).Equal("testing")
})
g.It("returns an error if the file does not exist", func() {
err := fs.Readfile("test.txt", buf)
g.Assert(err).IsNotNil()
g.Assert(errors.Is(err, os.ErrNotExist)).IsTrue()
})
g.It("returns an error if the \"file\" is a directory", func() {
err := os.Mkdir(filepath.Join(rfs.root, "/server/test.txt"), 0o755)
g.Assert(err).IsNil()
err = fs.Readfile("test.txt", buf)
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodeIsDirectory)).IsTrue()
})
g.It("cannot open a file outside the root directory", func() {
err := rfs.CreateServerFileFromString("/../test.txt", "testing")
g.Assert(err).IsNil()
err = fs.Readfile("/../test.txt", buf)
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
})
g.AfterEach(func() {
buf.Truncate(0)
atomic.StoreInt64(&fs.diskUsed, 0)
rfs.reset()
})
})
}
func TestFilesystem_Writefile(t *testing.T) { func TestFilesystem_Writefile(t *testing.T) {
g := Goblin(t) g := Goblin(t)
fs, rfs := NewFs() fs, rfs := NewFs()
@@ -101,10 +140,9 @@ func TestFilesystem_Writefile(t *testing.T) {
err := fs.Writefile("test.txt", r) err := fs.Writefile("test.txt", r)
g.Assert(err).IsNil() g.Assert(err).IsNil()
f, _, err := fs.File("test.txt") err = fs.Readfile("test.txt", buf)
g.Assert(err).IsNil() g.Assert(err).IsNil()
defer f.Close() g.Assert(buf.String()).Equal("test file content")
g.Assert(getFileContent(f)).Equal("test file content")
g.Assert(atomic.LoadInt64(&fs.diskUsed)).Equal(r.Size()) g.Assert(atomic.LoadInt64(&fs.diskUsed)).Equal(r.Size())
}) })
@@ -114,10 +152,9 @@ func TestFilesystem_Writefile(t *testing.T) {
err := fs.Writefile("/some/nested/test.txt", r) err := fs.Writefile("/some/nested/test.txt", r)
g.Assert(err).IsNil() g.Assert(err).IsNil()
f, _, err := fs.File("/some/nested/test.txt") err = fs.Readfile("/some/nested/test.txt", buf)
g.Assert(err).IsNil() g.Assert(err).IsNil()
defer f.Close() g.Assert(buf.String()).Equal("test file content")
g.Assert(getFileContent(f)).Equal("test file content")
}) })
g.It("can create a new file inside a nested directory without a trailing slash", func() { g.It("can create a new file inside a nested directory without a trailing slash", func() {
@@ -126,10 +163,9 @@ func TestFilesystem_Writefile(t *testing.T) {
err := fs.Writefile("some/../foo/bar/test.txt", r) err := fs.Writefile("some/../foo/bar/test.txt", r)
g.Assert(err).IsNil() g.Assert(err).IsNil()
f, _, err := fs.File("foo/bar/test.txt") err = fs.Readfile("foo/bar/test.txt", buf)
g.Assert(err).IsNil() g.Assert(err).IsNil()
defer f.Close() g.Assert(buf.String()).Equal("test file content")
g.Assert(getFileContent(f)).Equal("test file content")
}) })
g.It("cannot create a file outside the root directory", func() { g.It("cannot create a file outside the root directory", func() {
@@ -154,6 +190,28 @@ func TestFilesystem_Writefile(t *testing.T) {
g.Assert(IsErrorCode(err, ErrCodeDiskSpace)).IsTrue() g.Assert(IsErrorCode(err, ErrCodeDiskSpace)).IsTrue()
}) })
/*g.It("updates the total space used when a file is appended to", func() {
atomic.StoreInt64(&fs.diskUsed, 100)
b := make([]byte, 100)
_, _ = rand.Read(b)
r := bytes.NewReader(b)
err := fs.Writefile("test.txt", r)
g.Assert(err).IsNil()
g.Assert(atomic.LoadInt64(&fs.diskUsed)).Equal(int64(200))
// If we write less data than already exists, we should expect the total
// disk used to be decremented.
b = make([]byte, 50)
_, _ = rand.Read(b)
r = bytes.NewReader(b)
err = fs.Writefile("test.txt", r)
g.Assert(err).IsNil()
g.Assert(atomic.LoadInt64(&fs.diskUsed)).Equal(int64(150))
})*/
g.It("truncates the file when writing new contents", func() { g.It("truncates the file when writing new contents", func() {
r := bytes.NewReader([]byte("original data")) r := bytes.NewReader([]byte("original data"))
err := fs.Writefile("test.txt", r) err := fs.Writefile("test.txt", r)
@@ -163,10 +221,9 @@ func TestFilesystem_Writefile(t *testing.T) {
err = fs.Writefile("test.txt", r) err = fs.Writefile("test.txt", r)
g.Assert(err).IsNil() g.Assert(err).IsNil()
f, _, err := fs.File("test.txt") err = fs.Readfile("test.txt", buf)
g.Assert(err).IsNil() g.Assert(err).IsNil()
defer f.Close() g.Assert(buf.String()).Equal("new data")
g.Assert(getFileContent(f)).Equal("new data")
}) })
g.AfterEach(func() { g.AfterEach(func() {
@@ -508,80 +565,6 @@ func TestFilesystem_Delete(t *testing.T) {
} }
}) })
g.It("deletes a symlink but not it's target within the root directory", func() {
// Symlink to a file inside the root directory.
err := os.Symlink(filepath.Join(rfs.root, "server/source.txt"), filepath.Join(rfs.root, "server/symlink.txt"))
g.Assert(err).IsNil()
// Delete the symlink itself.
err = fs.Delete("symlink.txt")
g.Assert(err).IsNil()
// Ensure the symlink was deleted.
_, err = os.Lstat(filepath.Join(rfs.root, "server/symlink.txt"))
g.Assert(err).IsNotNil()
// Ensure the symlink target still exists.
_, err = os.Lstat(filepath.Join(rfs.root, "server/source.txt"))
g.Assert(err).IsNil()
})
g.It("does not delete files symlinked outside of the root directory", func() {
// Create a file outside the root directory.
err := rfs.CreateServerFileFromString("/../source.txt", "test content")
g.Assert(err).IsNil()
// Create a symlink to the file outside the root directory.
err = os.Symlink(filepath.Join(rfs.root, "source.txt"), filepath.Join(rfs.root, "/server/symlink.txt"))
g.Assert(err).IsNil()
// Delete the symlink. (This should pass as we will delete the symlink itself, not it's target)
err = fs.Delete("symlink.txt")
g.Assert(err).IsNil()
// Ensure the file outside the root directory still exists.
_, err = os.Lstat(filepath.Join(rfs.root, "source.txt"))
g.Assert(err).IsNil()
})
g.It("does not delete files symlinked through a directory outside of the root directory", func() {
// Create a directory outside the root directory.
err := os.Mkdir(filepath.Join(rfs.root, "foo"), 0o755)
g.Assert(err).IsNil()
// Create a file inside the directory that is outside the root.
err = rfs.CreateServerFileFromString("/../foo/source.txt", "test content")
g.Assert(err).IsNil()
// Symlink the directory that is outside the root to a file inside the root.
err = os.Symlink(filepath.Join(rfs.root, "foo"), filepath.Join(rfs.root, "server/symlink"))
g.Assert(err).IsNil()
// Delete a file inside the symlinked directory.
err = fs.Delete("symlink/source.txt")
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
// Ensure the file outside the root directory still exists.
_, err = os.Lstat(filepath.Join(rfs.root, "foo/source.txt"))
g.Assert(err).IsNil()
})
g.It("returns an error when trying to delete a non-existent file symlinked through a directory outside of the root directory", func() {
// Create a directory outside the root directory.
err := os.Mkdir(filepath.Join(rfs.root, "foo2"), 0o755)
g.Assert(err).IsNil()
// Symlink the directory that is outside the root to a file inside the root.
err = os.Symlink(filepath.Join(rfs.root, "foo2"), filepath.Join(rfs.root, "server/symlink"))
g.Assert(err).IsNil()
// Delete a file inside the symlinked directory.
err = fs.Delete("symlink/source.txt")
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
})
g.AfterEach(func() { g.AfterEach(func() {
rfs.reset() rfs.reset()

View File

@@ -2,7 +2,6 @@ package filesystem
import ( import (
"context" "context"
iofs "io/fs"
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
@@ -34,6 +33,8 @@ func (fs *Filesystem) IsIgnored(paths ...string) error {
// This logic is actually copied over from the SFTP server code. Ideally that eventually // This logic is actually copied over from the SFTP server code. Ideally that eventually
// either gets ported into this application, or is able to make use of this package. // either gets ported into this application, or is able to make use of this package.
func (fs *Filesystem) SafePath(p string) (string, error) { func (fs *Filesystem) SafePath(p string) (string, error) {
var nonExistentPathResolution string
// Start with a cleaned up path before checking the more complex bits. // Start with a cleaned up path before checking the more complex bits.
r := fs.unsafeFilePath(p) r := fs.unsafeFilePath(p)
@@ -43,24 +44,47 @@ func (fs *Filesystem) SafePath(p string) (string, error) {
if err != nil && !os.IsNotExist(err) { if err != nil && !os.IsNotExist(err) {
return "", errors.Wrap(err, "server/filesystem: failed to evaluate symlink") return "", errors.Wrap(err, "server/filesystem: failed to evaluate symlink")
} else if os.IsNotExist(err) { } else if os.IsNotExist(err) {
// The target of one of the symlinks (EvalSymlinks is recursive) does not exist. // The requested directory doesn't exist, so at this point we need to iterate up the
// So we get what target path does not exist and check if it's within the data // path chain until we hit a directory that _does_ exist and can be validated.
// directory. If it is, we return the original path, otherwise we return an error. parts := strings.Split(filepath.Dir(r), "/")
pErr, ok := err.(*iofs.PathError)
if !ok { var try string
return "", errors.Wrap(err, "server/filesystem: failed to evaluate symlink") // Range over all of the path parts and form directory pathings from the end
// moving up until we have a valid resolution or we run out of paths to try.
for k := range parts {
try = strings.Join(parts[:(len(parts)-k)], "/")
if !fs.unsafeIsInDataDirectory(try) {
break
}
t, err := filepath.EvalSymlinks(try)
if err == nil {
nonExistentPathResolution = t
break
}
} }
ep = pErr.Path }
// If the new path doesn't start with their root directory there is clearly an escape
// attempt going on, and we should NOT resolve this path for them.
if nonExistentPathResolution != "" {
if !fs.unsafeIsInDataDirectory(nonExistentPathResolution) {
return "", NewBadPathResolution(p, nonExistentPathResolution)
}
// If the nonExistentPathResolution variable is not empty then the initial path requested
// did not exist and we looped through the pathway until we found a match. At this point
// we've confirmed the first matched pathway exists in the root server directory, so we
// can go ahead and just return the path that was requested initially.
return r, nil
} }
// If the requested directory from EvalSymlinks begins with the server root directory go // If the requested directory from EvalSymlinks begins with the server root directory go
// ahead and return it. If not we'll return an error which will block any further action // ahead and return it. If not we'll return an error which will block any further action
// on the file. // on the file.
if fs.unsafeIsInDataDirectory(ep) { if fs.unsafeIsInDataDirectory(ep) {
// Returning the original path here instead of the resolved path ensures that return ep, nil
// whatever the user is trying to do will work as expected. If we returned the
// resolved path, the user would be unable to know that it is in fact a symlink.
return r, nil
} }
return "", NewBadPathResolution(p, r) return "", NewBadPathResolution(p, r)

View File

@@ -115,18 +115,20 @@ func TestFilesystem_Blocks_Symlinks(t *testing.T) {
panic(err) panic(err)
} }
if err := os.Symlink(filepath.Join(rfs.root, "malicious_does_not_exist.txt"), filepath.Join(rfs.root, "/server/symlinked_does_not_exist.txt")); err != nil {
panic(err)
}
if err := os.Symlink(filepath.Join(rfs.root, "/server/symlinked_does_not_exist.txt"), filepath.Join(rfs.root, "/server/symlinked_does_not_exist2.txt")); err != nil {
panic(err)
}
if err := os.Symlink(filepath.Join(rfs.root, "/malicious_dir"), filepath.Join(rfs.root, "/server/external_dir")); err != nil { if err := os.Symlink(filepath.Join(rfs.root, "/malicious_dir"), filepath.Join(rfs.root, "/server/external_dir")); err != nil {
panic(err) panic(err)
} }
g.Describe("Readfile", func() {
g.It("cannot read a file symlinked outside the root", func() {
b := bytes.Buffer{}
err := fs.Readfile("symlinked.txt", &b)
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
})
})
g.Describe("Writefile", func() { g.Describe("Writefile", func() {
g.It("cannot write to a file symlinked outside the root", func() { g.It("cannot write to a file symlinked outside the root", func() {
r := bytes.NewReader([]byte("testing")) r := bytes.NewReader([]byte("testing"))
@@ -136,22 +138,6 @@ func TestFilesystem_Blocks_Symlinks(t *testing.T) {
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue() g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
}) })
g.It("cannot write to a non-existent file symlinked outside the root", func() {
r := bytes.NewReader([]byte("testing what the fuck"))
err := fs.Writefile("symlinked_does_not_exist.txt", r)
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
})
g.It("cannot write to chained symlinks with target that does not exist outside the root", func() {
r := bytes.NewReader([]byte("testing what the fuck"))
err := fs.Writefile("symlinked_does_not_exist2.txt", r)
g.Assert(err).IsNotNil()
g.Assert(IsErrorCode(err, ErrCodePathResolution)).IsTrue()
})
g.It("cannot write a file to a directory symlinked outside the root", func() { g.It("cannot write a file to a directory symlinked outside the root", func() {
r := bytes.NewReader([]byte("testing")) r := bytes.NewReader([]byte("testing"))

View File

@@ -447,8 +447,8 @@ func (ip *InstallationProcess) Execute() (string, error) {
"compress": "false", "compress": "false",
}, },
}, },
Privileged: true,
NetworkMode: container.NetworkMode(config.Get().Docker.Network.Mode), NetworkMode: container.NetworkMode(config.Get().Docker.Network.Mode),
UsernsMode: container.UsernsMode(config.Get().Docker.UsernsMode),
} }
// Ensure the root directory for the server exists properly before attempting // Ensure the root directory for the server exists properly before attempting

View File

@@ -52,24 +52,6 @@ func (m *Manager) Client() remote.Client {
return m.client return m.client
} }
// Len returns the count of servers stored in the manager instance.
func (m *Manager) Len() int {
m.mu.RLock()
defer m.mu.RUnlock()
return len(m.servers)
}
// Keys returns all of the server UUIDs stored in the manager set.
func (m *Manager) Keys() []string {
m.mu.RLock()
defer m.mu.RUnlock()
keys := make([]string, len(m.servers))
for i, s := range m.servers {
keys[i] = s.ID()
}
return keys
}
// Put replaces all the current values in the collection with the value that // Put replaces all the current values in the collection with the value that
// is passed through. // is passed through.
func (m *Manager) Put(s []*Server) { func (m *Manager) Put(s []*Server) {

View File

@@ -1,58 +0,0 @@
package sftp
import (
"emperror.dev/errors"
"github.com/apex/log"
"github.com/pterodactyl/wings/internal/database"
"github.com/pterodactyl/wings/internal/models"
)
type eventHandler struct {
ip string
user string
server string
}
type FileAction struct {
// Entity is the targeted file or directory (depending on the event) that the action
// is being performed _against_, such as "/foo/test.txt". This will always be the full
// path to the element.
Entity string
// Target is an optional (often blank) field that only has a value in it when the event
// is specifically modifying the entity, such as a rename or move event. In that case
// the Target field will be the final value, such as "/bar/new.txt"
Target string
}
// Log parses a SFTP specific file activity event and then passes it off to be stored
// in the normal activity database.
func (eh *eventHandler) Log(e models.Event, fa FileAction) error {
metadata := map[string]interface{}{
"files": []string{fa.Entity},
}
if fa.Target != "" {
metadata["files"] = []map[string]string{
{"from": fa.Entity, "to": fa.Target},
}
}
a := models.Activity{
Server: eh.server,
Event: e,
Metadata: metadata,
IP: eh.ip,
}
if tx := database.Instance().Create(a.SetUser(eh.user)); tx.Error != nil {
return errors.WithStack(tx.Error)
}
return nil
}
// MustLog is a wrapper around log that will trigger a fatal error and exit the application
// if an error is encountered during the logging of the event.
func (eh *eventHandler) MustLog(e models.Event, fa FileAction) {
if err := eh.Log(e, fa); err != nil {
log.WithField("error", errors.WithStack(err)).WithField("event", e).Error("sftp: failed to log event")
}
}

View File

@@ -27,40 +27,32 @@ const (
) )
type Handler struct { type Handler struct {
mu sync.Mutex mu sync.Mutex
permissions []string
server *server.Server server *server.Server
fs *filesystem.Filesystem fs *filesystem.Filesystem
events *eventHandler
permissions []string
logger *log.Entry logger *log.Entry
ro bool ro bool
} }
// NewHandler returns a new connection handler for the SFTP server. This allows a given user // Returns a new connection handler for the SFTP server. This allows a given user
// to access the underlying filesystem. // to access the underlying filesystem.
func NewHandler(sc *ssh.ServerConn, srv *server.Server) (*Handler, error) { func NewHandler(sc *ssh.ServerConn, srv *server.Server) *Handler {
uuid, ok := sc.Permissions.Extensions["user"]
if !ok {
return nil, errors.New("sftp: mismatched Wings and Panel versions — Panel 1.10 is required for this version of Wings.")
}
events := eventHandler{
ip: sc.RemoteAddr().String(),
user: uuid,
server: srv.ID(),
}
return &Handler{ return &Handler{
permissions: strings.Split(sc.Permissions.Extensions["permissions"], ","), permissions: strings.Split(sc.Permissions.Extensions["permissions"], ","),
server: srv, server: srv,
fs: srv.Filesystem(), fs: srv.Filesystem(),
events: &events,
ro: config.Get().System.Sftp.ReadOnly, ro: config.Get().System.Sftp.ReadOnly,
logger: log.WithFields(log.Fields{"subsystem": "sftp", "user": uuid, "ip": sc.RemoteAddr()}), logger: log.WithFields(log.Fields{
}, nil "subsystem": "sftp",
"username": sc.User(),
"ip": sc.RemoteAddr(),
}),
}
} }
// Handlers returns the sftp.Handlers for this struct. // Returns the sftp.Handlers for this struct.
func (h *Handler) Handlers() sftp.Handlers { func (h *Handler) Handlers() sftp.Handlers {
return sftp.Handlers{ return sftp.Handlers{
FileGet: h, FileGet: h,
@@ -129,16 +121,11 @@ func (h *Handler) Filewrite(request *sftp.Request) (io.WriterAt, error) {
} }
// Chown may or may not have been called in the touch function, so always do // Chown may or may not have been called in the touch function, so always do
// it at this point to avoid the file being improperly owned. // it at this point to avoid the file being improperly owned.
_ = h.fs.Chown(request.Filepath) _ = h.server.Filesystem().Chown(request.Filepath)
event := server.ActivitySftpWrite
if permission == PermissionFileCreate {
event = server.ActivitySftpCreate
}
h.events.MustLog(event, FileAction{Entity: request.Filepath})
return f, nil return f, nil
} }
// Filecmd hander for basic SFTP system calls related to files, but not anything to do with reading // Filecmd handler for basic SFTP system calls related to files, but not anything to do with reading
// or writing to those files. // or writing to those files.
func (h *Handler) Filecmd(request *sftp.Request) error { func (h *Handler) Filecmd(request *sftp.Request) error {
if h.ro { if h.ro {
@@ -185,7 +172,6 @@ func (h *Handler) Filecmd(request *sftp.Request) error {
l.WithField("error", err).Error("failed to rename file") l.WithField("error", err).Error("failed to rename file")
return sftp.ErrSSHFxFailure return sftp.ErrSSHFxFailure
} }
h.events.MustLog(server.ActivitySftpRename, FileAction{Entity: request.Filepath, Target: request.Target})
break break
// Handle deletion of a directory. This will properly delete all of the files and // Handle deletion of a directory. This will properly delete all of the files and
// folders within that directory if it is not already empty (unlike a lot of SFTP // folders within that directory if it is not already empty (unlike a lot of SFTP
@@ -194,12 +180,10 @@ func (h *Handler) Filecmd(request *sftp.Request) error {
if !h.can(PermissionFileDelete) { if !h.can(PermissionFileDelete) {
return sftp.ErrSSHFxPermissionDenied return sftp.ErrSSHFxPermissionDenied
} }
p := filepath.Clean(request.Filepath) if err := h.fs.Delete(request.Filepath); err != nil {
if err := h.fs.Delete(p); err != nil {
l.WithField("error", err).Error("failed to remove directory") l.WithField("error", err).Error("failed to remove directory")
return sftp.ErrSSHFxFailure return sftp.ErrSSHFxFailure
} }
h.events.MustLog(server.ActivitySftpDelete, FileAction{Entity: request.Filepath})
return sftp.ErrSSHFxOk return sftp.ErrSSHFxOk
// Handle requests to create a new Directory. // Handle requests to create a new Directory.
case "Mkdir": case "Mkdir":
@@ -207,12 +191,11 @@ func (h *Handler) Filecmd(request *sftp.Request) error {
return sftp.ErrSSHFxPermissionDenied return sftp.ErrSSHFxPermissionDenied
} }
name := strings.Split(filepath.Clean(request.Filepath), "/") name := strings.Split(filepath.Clean(request.Filepath), "/")
p := strings.Join(name[0:len(name)-1], "/") err := h.fs.CreateDirectory(name[len(name)-1], strings.Join(name[0:len(name)-1], "/"))
if err := h.fs.CreateDirectory(name[len(name)-1], p); err != nil { if err != nil {
l.WithField("error", err).Error("failed to create directory") l.WithField("error", err).Error("failed to create directory")
return sftp.ErrSSHFxFailure return sftp.ErrSSHFxFailure
} }
h.events.MustLog(server.ActivitySftpCreateDirectory, FileAction{Entity: request.Filepath})
break break
// Support creating symlinks between files. The source and target must resolve within // Support creating symlinks between files. The source and target must resolve within
// the server home directory. // the server home directory.
@@ -245,7 +228,6 @@ func (h *Handler) Filecmd(request *sftp.Request) error {
l.WithField("error", err).Error("failed to remove a file") l.WithField("error", err).Error("failed to remove a file")
return sftp.ErrSSHFxFailure return sftp.ErrSSHFxFailure
} }
h.events.MustLog(server.ActivitySftpDelete, FileAction{Entity: request.Filepath})
return sftp.ErrSSHFxOk return sftp.ErrSSHFxOk
default: default:
return sftp.ErrSSHFxOpUnsupported return sftp.ErrSSHFxOpUnsupported
@@ -305,10 +287,15 @@ func (h *Handler) can(permission string) bool {
if h.server.IsSuspended() { if h.server.IsSuspended() {
return false return false
} }
// SFTPServer owners and super admins have their permissions returned as '[*]' via the Panel
// API, so for the sake of speed do an initial check for that before iterating over the
// entire array of permissions.
if len(h.permissions) == 1 && h.permissions[0] == "*" {
return true
}
for _, p := range h.permissions { for _, p := range h.permissions {
// If we match the permission specifically, or the user has been granted the "*" if p == permission {
// permission because they're an admin, let them through.
if p == permission || p == "*" {
return true return true
} }
} }

View File

@@ -1,11 +1,17 @@
package sftp package sftp
import ( import (
"bytes"
"context" "context"
"crypto/ecdsa"
"crypto/ed25519"
"crypto/elliptic"
"crypto/rand" "crypto/rand"
"crypto/rsa"
"crypto/x509" "crypto/x509"
"encoding/pem" "encoding/pem"
"io" "io"
"io/ioutil"
"net" "net"
"os" "os"
"path" "path"
@@ -16,7 +22,6 @@ import (
"emperror.dev/errors" "emperror.dev/errors"
"github.com/apex/log" "github.com/apex/log"
"github.com/pkg/sftp" "github.com/pkg/sftp"
"golang.org/x/crypto/ed25519"
"golang.org/x/crypto/ssh" "golang.org/x/crypto/ssh"
"github.com/pterodactyl/wings/config" "github.com/pterodactyl/wings/config"
@@ -51,61 +56,45 @@ func New(m *server.Manager) *SFTPServer {
// SFTP connections. This will automatically generate an ED25519 key if one does // SFTP connections. This will automatically generate an ED25519 key if one does
// not already exist on the system for host key verification purposes. // not already exist on the system for host key verification purposes.
func (c *SFTPServer) Run() error { func (c *SFTPServer) Run() error {
if _, err := os.Stat(c.PrivateKeyPath()); os.IsNotExist(err) { keys, err := c.loadPrivateKeys()
if err := c.generateED25519PrivateKey(); err != nil {
return err
}
} else if err != nil {
return errors.Wrap(err, "sftp: could not stat private key file")
}
pb, err := os.ReadFile(c.PrivateKeyPath())
if err != nil {
return errors.Wrap(err, "sftp: could not read private key file")
}
private, err := ssh.ParsePrivateKey(pb)
if err != nil { if err != nil {
return err return err
} }
conf := &ssh.ServerConfig{ conf := &ssh.ServerConfig{
NoClientAuth: false, NoClientAuth: false,
MaxAuthTries: 6, MaxAuthTries: 6,
PasswordCallback: func(conn ssh.ConnMetadata, password []byte) (*ssh.Permissions, error) { PasswordCallback: c.passwordCallback,
return c.makeCredentialsRequest(conn, remote.SftpAuthPassword, string(password)) PublicKeyCallback: c.publicKeyCallback,
}, }
PublicKeyCallback: func(conn ssh.ConnMetadata, key ssh.PublicKey) (*ssh.Permissions, error) {
return c.makeCredentialsRequest(conn, remote.SftpAuthPublicKey, string(ssh.MarshalAuthorizedKey(key))) for _, k := range keys {
}, conf.AddHostKey(k)
} }
conf.AddHostKey(private)
listener, err := net.Listen("tcp", c.Listen) listener, err := net.Listen("tcp", c.Listen)
if err != nil { if err != nil {
return err return err
} }
public := string(ssh.MarshalAuthorizedKey(private.PublicKey())) log.WithField("listen", c.Listen).Info("sftp server listening for connections")
log.WithField("listen", c.Listen).WithField("public_key", strings.Trim(public, "\n")).Info("sftp server listening for connections")
for { for {
if conn, _ := listener.Accept(); conn != nil { if conn, _ := listener.Accept(); conn != nil {
go func(conn net.Conn) { go func(conn net.Conn) {
defer conn.Close() defer conn.Close()
if err := c.AcceptInbound(conn, conf); err != nil { c.AcceptInbound(conn, conf)
log.WithField("error", err).Error("sftp: failed to accept inbound connection")
}
}(conn) }(conn)
} }
} }
} }
// AcceptInbound handles an inbound connection to the instance and determines if we should // Handles an inbound connection to the instance and determines if we should serve the
// serve the request or not. // request or not.
func (c *SFTPServer) AcceptInbound(conn net.Conn, config *ssh.ServerConfig) error { func (c *SFTPServer) AcceptInbound(conn net.Conn, config *ssh.ServerConfig) {
// Before beginning a handshake must be performed on the incoming net.Conn // Before beginning a handshake must be performed on the incoming net.Conn
sconn, chans, reqs, err := ssh.NewServerConn(conn, config) sconn, chans, reqs, err := ssh.NewServerConn(conn, config)
if err != nil { if err != nil {
return errors.WithStack(err) return
} }
defer sconn.Close() defer sconn.Close()
go ssh.DiscardRequests(reqs) go ssh.DiscardRequests(reqs)
@@ -114,7 +103,7 @@ func (c *SFTPServer) AcceptInbound(conn net.Conn, config *ssh.ServerConfig) erro
// If its not a session channel we just move on because its not something we // If its not a session channel we just move on because its not something we
// know how to handle at this point. // know how to handle at this point.
if ch.ChannelType() != "session" { if ch.ChannelType() != "session" {
ch.Reject(ssh.UnknownChannelType, "unknown channel type") _ = ch.Reject(ssh.UnknownChannelType, "unknown channel type")
continue continue
} }
@@ -128,7 +117,7 @@ func (c *SFTPServer) AcceptInbound(conn net.Conn, config *ssh.ServerConfig) erro
// Channels have a type that is dependent on the protocol. For SFTP // Channels have a type that is dependent on the protocol. For SFTP
// this is "subsystem" with a payload that (should) be "sftp". Discard // this is "subsystem" with a payload that (should) be "sftp". Discard
// anything else we receive ("pty", "shell", etc) // anything else we receive ("pty", "shell", etc)
req.Reply(req.Type == "subsystem" && string(req.Payload[4:]) == "sftp", nil) _ = req.Reply(req.Type == "subsystem" && string(req.Payload[4:]) == "sftp", nil)
} }
}(requests) }(requests)
@@ -146,61 +135,180 @@ func (c *SFTPServer) AcceptInbound(conn net.Conn, config *ssh.ServerConfig) erro
return s.ID() == uuid return s.ID() == uuid
}) })
if srv == nil { if srv == nil {
_ = conn.Close()
continue continue
} }
// Spin up a SFTP server instance for the authenticated user's server allowing // Spin up a SFTP server instance for the authenticated user's server allowing
// them access to the underlying filesystem. // them access to the underlying filesystem.
handler, err := NewHandler(sconn, srv) handler := sftp.NewRequestServer(channel, NewHandler(sconn, srv).Handlers())
if err != nil { if err := handler.Serve(); err == io.EOF {
return errors.WithStackIf(err) _ = handler.Close()
}
rs := sftp.NewRequestServer(channel, handler.Handlers())
if err := rs.Serve(); err == io.EOF {
_ = rs.Close()
} }
} }
return nil
} }
// Generates a new ED25519 private key that is used for host authentication when func (c *SFTPServer) loadPrivateKeys() ([]ssh.Signer, error) {
// a user connects to the SFTP server. if _, err := os.Stat(path.Join(c.BasePath, ".sftp/id_rsa")); err != nil {
func (c *SFTPServer) generateED25519PrivateKey() error { if !os.IsNotExist(err) {
_, priv, err := ed25519.GenerateKey(rand.Reader) return nil, err
if err != nil { }
return errors.Wrap(err, "sftp: failed to generate ED25519 private key")
if err := c.generateRSAPrivateKey(); err != nil {
return nil, err
}
} }
if err := os.MkdirAll(path.Dir(c.PrivateKeyPath()), 0o755); err != nil { rsaBytes, err := ioutil.ReadFile(path.Join(c.BasePath, ".sftp/id_rsa"))
return errors.Wrap(err, "sftp: could not create internal sftp data directory")
}
o, err := os.OpenFile(c.PrivateKeyPath(), os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil { if err != nil {
return errors.WithStack(err) return nil, errors.Wrap(err, "sftp/server: could not read private key file")
}
rsaPrivateKey, err := ssh.ParsePrivateKey(rsaBytes)
if err != nil {
return nil, err
}
if _, err := os.Stat(path.Join(c.BasePath, ".sftp/id_ecdsa")); err != nil {
if !os.IsNotExist(err) {
return nil, err
}
if err := c.generateECDSAPrivateKey(); err != nil {
return nil, err
}
}
ecdsaBytes, err := ioutil.ReadFile(path.Join(c.BasePath, ".sftp/id_ecdsa"))
if err != nil {
return nil, errors.Wrap(err, "sftp/server: could not read private key file")
}
ecdsaPrivateKey, err := ssh.ParsePrivateKey(ecdsaBytes)
if err != nil {
return nil, err
}
if _, err := os.Stat(path.Join(c.BasePath, ".sftp/id_ed25519")); err != nil {
if !os.IsNotExist(err) {
return nil, err
}
if err := c.generateEd25519PrivateKey(); err != nil {
return nil, err
}
}
ed25519Bytes, err := ioutil.ReadFile(path.Join(c.BasePath, ".sftp/id_ed25519"))
if err != nil {
return nil, errors.Wrap(err, "sftp/server: could not read private key file")
}
ed25519PrivateKey, err := ssh.ParsePrivateKey(ed25519Bytes)
if err != nil {
return nil, err
}
return []ssh.Signer{
rsaPrivateKey,
ecdsaPrivateKey,
ed25519PrivateKey,
}, nil
}
// generateRSAPrivateKey generates a RSA-4096 private key that will be used by the SFTP server.
func (c *SFTPServer) generateRSAPrivateKey() error {
key, err := rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
return err
}
if err := os.MkdirAll(path.Dir(c.PrivateKeyPath("rsa")), 0o755); err != nil {
return errors.Wrap(err, "sftp/server: could not create .sftp directory")
}
o, err := os.OpenFile(c.PrivateKeyPath("rsa"), os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
return err
} }
defer o.Close() defer o.Close()
b, err := x509.MarshalPKCS8PrivateKey(priv) if err := pem.Encode(o, &pem.Block{
if err != nil { Type: "RSA PRIVATE KEY",
return errors.Wrap(err, "sftp: failed to marshal private key into bytes") Bytes: x509.MarshalPKCS1PrivateKey(key),
} }); err != nil {
if err := pem.Encode(o, &pem.Block{Type: "PRIVATE KEY", Bytes: b}); err != nil { return err
return errors.Wrap(err, "sftp: failed to write ED25519 private key to disk")
} }
return nil return nil
} }
func (c *SFTPServer) makeCredentialsRequest(conn ssh.ConnMetadata, t remote.SftpAuthRequestType, p string) (*ssh.Permissions, error) { // generateECDSAPrivateKey generates a ECDSA-P256 private key that will be used by the SFTP server.
func (c *SFTPServer) generateECDSAPrivateKey() error {
key, err := ecdsa.GenerateKey(elliptic.P521(), rand.Reader)
if err != nil {
return err
}
if err := os.MkdirAll(path.Dir(c.PrivateKeyPath("ecdsa")), 0o755); err != nil {
return errors.Wrap(err, "sftp/server: could not create .sftp directory")
}
o, err := os.OpenFile(c.PrivateKeyPath("ecdsa"), os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
return err
}
defer o.Close()
privBytes, err := x509.MarshalPKCS8PrivateKey(key)
if err != nil {
return err
}
if err := pem.Encode(o, &pem.Block{
Type: "PRIVATE KEY",
Bytes: privBytes,
}); err != nil {
return err
}
return nil
}
// generateEd25519PrivateKey generates an ed25519 private key that will be used by the SFTP server.
func (c *SFTPServer) generateEd25519PrivateKey() error {
_, key, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
return err
}
if err := os.MkdirAll(path.Dir(c.PrivateKeyPath("ed25519")), 0o755); err != nil {
return errors.Wrap(err, "sftp/server: could not create .sftp directory")
}
o, err := os.OpenFile(c.PrivateKeyPath("ed25519"), os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
return err
}
defer o.Close()
privBytes, err := x509.MarshalPKCS8PrivateKey(key)
if err != nil {
return err
}
if err := pem.Encode(o, &pem.Block{
Type: "PRIVATE KEY",
Bytes: privBytes,
}); err != nil {
return err
}
return nil
}
// PrivateKeyPath returns the path the host private key for this server instance.
func (c *SFTPServer) PrivateKeyPath(name string) string {
return path.Join(c.BasePath, ".sftp", "id_"+name)
}
// A function capable of validating user credentials with the Panel API.
func (c *SFTPServer) passwordCallback(conn ssh.ConnMetadata, pass []byte) (*ssh.Permissions, error) {
request := remote.SftpAuthRequest{ request := remote.SftpAuthRequest{
Type: t,
User: conn.User(), User: conn.User(),
Pass: p, Pass: string(pass),
IP: conn.RemoteAddr().String(), IP: conn.RemoteAddr().String(),
SessionID: conn.SessionID(), SessionID: conn.SessionID(),
ClientVersion: conn.ClientVersion(), ClientVersion: conn.ClientVersion(),
Type: "password",
} }
logger := log.WithFields(log.Fields{"subsystem": "sftp", "method": request.Type, "username": request.User, "ip": request.IP}) logger := log.WithFields(log.Fields{"subsystem": "sftp", "username": conn.User(), "ip": conn.RemoteAddr().String()})
logger.Debug("validating credentials for SFTP connection") logger.Debug("validating credentials for SFTP connection")
if !validUsernameRegexp.MatchString(request.User) { if !validUsernameRegexp.MatchString(request.User) {
@@ -208,6 +316,51 @@ func (c *SFTPServer) makeCredentialsRequest(conn ssh.ConnMetadata, t remote.Sftp
return nil, &remote.SftpInvalidCredentialsError{} return nil, &remote.SftpInvalidCredentialsError{}
} }
if len(pass) < 1 {
logger.Warn("failed to validate user credentials (invalid format)")
return nil, &remote.SftpInvalidCredentialsError{}
}
resp, err := c.manager.Client().ValidateSftpCredentials(context.Background(), request)
if err != nil {
if _, ok := err.(*remote.SftpInvalidCredentialsError); ok {
logger.Warn("failed to validate user credentials (invalid username or password)")
} else {
logger.WithField("error", err).Error("encountered an error while trying to validate user credentials")
}
return nil, err
}
logger.WithField("server", resp.Server).Debug("credentials validated and matched to server instance")
sshPerm := &ssh.Permissions{
Extensions: map[string]string{
"uuid": resp.Server,
"user": conn.User(),
"permissions": strings.Join(resp.Permissions, ","),
},
}
return sshPerm, nil
}
func (c *SFTPServer) publicKeyCallback(conn ssh.ConnMetadata, key ssh.PublicKey) (*ssh.Permissions, error) {
request := remote.SftpAuthRequest{
User: conn.User(),
Pass: "KEKW",
IP: conn.RemoteAddr().String(),
SessionID: conn.SessionID(),
ClientVersion: conn.ClientVersion(),
Type: "publicKey",
}
logger := log.WithFields(log.Fields{"subsystem": "sftp", "username": conn.User(), "ip": conn.RemoteAddr().String()})
logger.Debug("validating public key for SFTP connection")
if !validUsernameRegexp.MatchString(request.User) {
logger.Warn("failed to validate user credentials (invalid format)")
return nil, &remote.SftpInvalidCredentialsError{}
}
resp, err := c.manager.Client().ValidateSftpCredentials(context.Background(), request) resp, err := c.manager.Client().ValidateSftpCredentials(context.Background(), request)
if err != nil { if err != nil {
if _, ok := err.(*remote.SftpInvalidCredentialsError); ok { if _, ok := err.(*remote.SftpInvalidCredentialsError); ok {
@@ -218,20 +371,27 @@ func (c *SFTPServer) makeCredentialsRequest(conn ssh.ConnMetadata, t remote.Sftp
return nil, err return nil, err
} }
logger.WithField("server", resp.Server).Debug("credentials validated and matched to server instance") if len(resp.SSHKeys) < 1 {
permissions := ssh.Permissions{ return nil, &remote.SftpInvalidCredentialsError{}
Extensions: map[string]string{
"ip": conn.RemoteAddr().String(),
"uuid": resp.Server,
"user": resp.User,
"permissions": strings.Join(resp.Permissions, ","),
},
} }
return &permissions, nil for _, k := range resp.SSHKeys {
} storedPublicKey, _, _, _, err := ssh.ParseAuthorizedKey([]byte(k))
if err != nil {
return nil, err
}
// PrivateKeyPath returns the path the host private key for this server instance. if !bytes.Equal(key.Marshal(), storedPublicKey.Marshal()) {
func (c *SFTPServer) PrivateKeyPath() string { continue
return path.Join(c.BasePath, ".sftp/id_ed25519") }
return &ssh.Permissions{
Extensions: map[string]string{
"uuid": resp.Server,
"user": conn.User(),
"permissions": strings.Join(resp.Permissions, ","),
},
}, nil
}
return nil, &remote.SftpInvalidCredentialsError{}
} }

View File

@@ -23,7 +23,7 @@ type SinkPool struct {
} }
// NewSinkPool returns a new empty SinkPool. A sink pool generally lives with a // NewSinkPool returns a new empty SinkPool. A sink pool generally lives with a
// server instance for its full lifetime. // server instance for it's full lifetime.
func NewSinkPool() *SinkPool { func NewSinkPool() *SinkPool {
return &SinkPool{} return &SinkPool{}
} }

View File

@@ -1,29 +0,0 @@
package system
import (
"math/rand"
"regexp"
"strings"
)
var ipTrimRegex = regexp.MustCompile(`(:\d*)?$`)
const characters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890"
// RandomString generates a random string of alpha-numeric characters using a
// pseudo-random number generator. The output of this function IS NOT cryptographically
// secure, it is used solely for generating random strings outside a security context.
func RandomString(n int) string {
var b strings.Builder
b.Grow(n)
for i := 0; i < n; i++ {
b.WriteByte(characters[rand.Intn(len(characters))])
}
return b.String()
}
// TrimIPSuffix removes the internal port value from an IP address to ensure we're only
// ever working directly with the IP address.
func TrimIPSuffix(s string) string {
return ipTrimRegex.ReplaceAllString(s, "")
}

View File

@@ -2,16 +2,8 @@ package main
import ( import (
"github.com/pterodactyl/wings/cmd" "github.com/pterodactyl/wings/cmd"
"math/rand"
"time"
) )
func main() { func main() {
// Since we make use of the math/rand package in the code, especially for generating
// non-cryptographically secure random strings we need to seed the RNG. Just make use
// of the current time for this.
rand.Seed(time.Now().UnixNano())
// Execute the main binary code.
cmd.Execute() cmd.Execute()
} }