diff --git a/ci/vale/dictionary.txt b/ci/vale/dictionary.txt index bc6ded96b76..22417cf928a 100644 --- a/ci/vale/dictionary.txt +++ b/ci/vale/dictionary.txt @@ -2498,6 +2498,7 @@ teamspeak teardown techdocs techland +Tectia telecom telehealth teleporting diff --git a/docs/contributors/leon-yen/_index.md b/docs/contributors/leon-yen/_index.md new file mode 100644 index 00000000000..f0837849e74 --- /dev/null +++ b/docs/contributors/leon-yen/_index.md @@ -0,0 +1,8 @@ +--- +title: "Leon Yen" +link: "https://github.com/leonski-ts2/" +email: "leon@threatspan.com" +description: "The Linode documentation library's profile page and submission listing for Leon Yen" +--- + +Leon Yen is an independent contributor to the Linode documentation library. Their submissions can be found listed below. \ No newline at end of file diff --git a/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/high-impact-ad-traffic-arch-1.svg b/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/high-impact-ad-traffic-arch-1.svg new file mode 100644 index 00000000000..fcf65fdb888 --- /dev/null +++ b/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/high-impact-ad-traffic-arch-1.svg @@ -0,0 +1,395 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/index.md b/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/index.md index 23a60e5ef2d..c1dd658bf5e 100644 --- a/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/index.md +++ b/docs/guides/akamai/solutions/preparing-infrastructure-for-high-impact-ad-traffic/index.md @@ -62,19 +62,20 @@ At the edge, Akamai’s [App & API Protector](https://www.akamai.com/products/ap 1. Edge servers running [Ion](https://www.akamai.com/products/web-performance-optimization) deliver web content to end users, reduce DNS lookup time, and accelerate traffic to provide a fast and responsive user experience. -1. [Image & Video Manager](https://www.akamai.com/products/image-and-video-manager) provides image optimization to increase performance while maintaining visible quality. +2. [Image & Video Manager](https://www.akamai.com/products/image-and-video-manager) provides image optimization to increase performance while maintaining visible quality. -1. Edge servers using App & API Protector and Bot Manager protect against established and emerging threats including DDsS, bot attacks, application and API attacks, and more. +3. Edge servers using App & API Protector and Bot Manager protect against established and emerging threats including DDsS, bot attacks, application and API attacks, and more. -1. EdgeWorkers allow performance-sensitive compute workloads to be moved to the edge to reduce both latency and origin load. +4. EdgeWorkers allow performance-sensitive compute workloads to be moved to the edge to reduce both latency and origin load. -1. Queue-it waiting rooms allow for user prioritization and enhance the user experience during periods of extremely high traffic. +5. Queue-it waiting rooms allow for user prioritization and enhance the user experience during periods of extremely high traffic. -1. TrafficPeak on Akamai Connected Cloud provides near real-time visualization of event data. +6. TrafficPeak on Akamai Connected Cloud provides near real-time visualization of event data. -1. Akamai routes CDN traffic through designated edge servers. This allows customers to drop traffic from other sources and prevent attackers from bypassing edge-based protection. +7. Akamai routes CDN traffic through designated edge servers. This allows customers to drop traffic from other sources and prevent attackers from bypassing edge-based protection. +{#high-impact-ad-arch .large-diagram} -![High-Impact Advertising Infrastructure Architecture](high-impact-ad-traffic-architecture-1.jpg "High-Impact Advertising Infrastructure Architecture") +![High-Impact Advertising Infrastructure Architecture](high-impact-ad-traffic-arch-1.svg?diagram-description-id=high-impact-ad-arch) ### Systems and Components diff --git a/docs/guides/tools-reference/file-transfer/comparing-data-transfer-utilities/index.md b/docs/guides/tools-reference/file-transfer/comparing-data-transfer-utilities/index.md new file mode 100644 index 00000000000..3b9eae2405c --- /dev/null +++ b/docs/guides/tools-reference/file-transfer/comparing-data-transfer-utilities/index.md @@ -0,0 +1,126 @@ +--- +slug: comparing-data-transfer-utilities +title: "Comparing Data Transfer Utilities" +description: "This guide presents and compares various data transfer utilities and protocols, including rclone, rsync, SCP, SFTP, and FTP/SFTP clients." +authors: ["Leon Yen"] +contributors: ["Leon Yen"] +published: 2024-10-09 +keywords: ['data transfer utility','data transfer','file transfer'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +external_resources: +- '[Introduction to Rsync](/docs/guides/introduction-to-rsync/)' +- '[Transfer Files With the scp Command on Linux](/docs/guides/how-to-use-scp/)' +- '[Transfer Files with SFTP](/docs/guides/sftp-linux/)' +- '[What Is FTP and How Does It Work?](/docs/guides/what-is-ftp/)' +- '[Transfer Files with FileZilla](/docs/guides/filezilla/)' +--- + +Technology professionals and end users rely on a myriad of utilities for moving data between locations. This guide discusses several of these data transfer protocols and utilities: + +- [rsync](#rsync) +- [rclone](#rclone) +- [SCP](#scp) +- [SFTP](#sftp) +- [FTP/SFTP clients](#ftpsftp-clients) + +Details of each utility's core features, use cases, key considerations, strengths, and limitations are described. + +## What Is a Data Transfer Utility? + +A data transfer utility’s primary purpose is to facilitate the movement of data between different systems/devices, networks, and IT environments. At its core, a data transfer utility must transmit data from origin to destination efficiently and securely, with minimal disruption and errors. Depending on the use case, different data transfer tools may work better than others. + +Common data transfer scenarios range from simple file transfers and sharing to full-scale system backups, real-time data replication, and data migrations between different on-premise or cloud environments. Each use case often benefits from a different type of data transfer utility. + +## Transfer Considerations + +The following are some general transfer considerations to keep in mind when selecting a data transfer utility for a particular use case or scenario. + +### Egress and Ingress Cost + +Copying a single file from a desktop client to a cloud server calls for a different data transfer approach than migrating large volumes of data at scale between enterprise IT environments. When evaluating data transfer utilities, you should consider relative egress and ingress costs and the volume of data being transferred. + +### Data Storage Size and Type + +Along with how much data is being transferred, the kind of data and the destination are equally important (e.g., your data’s source and target destination, their respective file systems, and how much storage is required once the data is at rest). + +Most end users only store data in traditionally structured file systems when using their local environment, with files organized hierarchically into folders. These file systems are implemented on top of *block storage* devices. Block storage devices are an abstraction created by an operating system for the hardware of a hard drive. Block storage is also offered as a product by cloud platforms, where these devices can be *attached* to a compute instance, and the user can install a file system on them. + +Cloud platforms also offer an alternative service called *object storage*. Object storage saves files in a non-hierarchical, self-contained “flat” format. This allows for increased scalability and sizable reductions in cost and complexity. + +### File Type and Size + +Different file types and sizes also call for different data transfer utilities. For example, server log files grow in size quickly but can be easily compressed, whereas media files (e.g., MP4 video files, WAV sounds files) are usually quite large, aren’t easily compressed, and typically do not change once saved. Your choice of data transfer utility will depend on what kind of files are being transferred. + +### One-time Transfer vs. Recurring Syncs + +A specific utility may be well-suited for your use case depending on whether the data transfer is a single event or a recurring job. For example, an ad-hoc file transfer to a cloud server may only require a desktop-based data transfer utility, while a regularly scheduled backup job may require an automated tool or process that works at scale. + +## Comparing Data Transfer Utilities + +### Rsync + +[Rsync](https://rsync.samba.org/) is a standard Linux-based data transfer utility designed for syncing files between remote and local servers. Known for its flexibility, speed, and scriptable configuration, rsync is often a go-to utility for Linux administrators looking to quickly transfer and synchronize files between systems. rsync is used to transfer data between file systems and does not interact with object storage. + +#### Use Cases + +Rsync is ideal for syncing files between local and remote Linux machines in incremental backup and transfer use cases, as well as customizing large and complicated sync jobs with specific options and settings. rsync can also be used to sync files between two directories that are both stored locally. Its delta-transfer algorithm reduces network traffic during syncs by only sending parts of a file that differ from the files on the recipient machine. + +#### Pros and Cons + +Rsync excels in use cases that require a proven method for transferring files and situations that call for scheduling and automation capabilities, such as using rsync with Linux cron jobs. However, it also requires familiarity with the command line. + +### Rclone + +[Rclone](https://rclone.org/) is a command-line utility for syncing files and directories to and from different cloud storage providers, servers, and workstations. A number of [rclone's commands](https://rclone.org/commands/) are similar to common Linux commands (e.g., cp, mv, mount, ls), so it can feel familiar to use. Rclone was designed for file transfers between cloud servers and on-premises servers and workstations, whereas rsync can be configured for more sophisticated file synchronization capabilities. Rclone can transfer data to and from both file systems and object storage. + +#### Use Cases + +Rclone is used for basic cloud synchronization and the copying and managing files in the cloud. For example, if you’re copying database files or directories from a cloud server to an on-premises system, rclone is an ideal data transfer utility for the job. + +#### Pros and Cons + +Rclone supports leading cloud services like Akamai, Amazon S3, Microsoft OneDrive, Google Drive/Cloud Storage, Microsoft Azure Blob/File Storage, DropBox, and more. Rclone can also recover from interrupted connections during data transfers. As with rsync, rclone can perform a sync operation, but it also supports a [bidirectional sync](https://rclone.org/commands/rclone_bisync/) operation. + +### SCP + +[Secure copy (SCP)](https://en.wikipedia.org/wiki/Secure_copy_protocol) is a Linux system command for securely copying files between hosts. SCP was designed to be a more secure version of remote copy (RCP), relying on the secure shell (SSH) protocol to encrypt network traffic between data transfers. SCP is used to transfer data between file systems and does not interact with object storage. + +#### Use cases + +SCP is ideal for general file transfer use cases that require copying files from a local computer to a remote server, or from remote servers to a local computer, in a secure and resilient manner. Unlike rsync, SCP is not used to copy files between directories that are both local. + +#### Pros and Cons + +Some users may find SCP to be less complicated to use than rsync. SCP is not capable of restarting transfers as rsync can, because it lacks a similar delta-transfer algorithm. + +### SFTP + +Like SCP, [secure file transfer protocol (SFTP)](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol) is commonly used for transferring files over an SSH-encrypted channel. However, SFTP can be used with a range of interactive commands (e.g, creating/deleting directories and files, settings permissions), while SCP is used for transferring files in a non-interactive manner. SFTP is used to transfer files between file systems and is generally not natively supported by object storage services, but some services may expose a compatibility layer that allows SFTP access. + +#### Use Cases + +SFTP is ideal when strong file encryption is required or mandated for transferring files over the internet. For example, compliance regulations like HIPAA, PCI-DSS, and GDPR require the use of SFTP to meet data integrity, confidentiality, and security requirements for safe data transmission. SFTP is widely regarded as the secure replacement for FTP. + +#### Pros and Cons + +SFTP works like traditional FTP, but over an encrypted, secure connection. SFTP supports both username and password and SSH key authentication, with some SFTP clients also supporting MFA and role-defined access. When using an SFTP client, it’s important to make sure it's not configured to use outdated encryption protocols like MD5 or DES (versus AES-128 or AES-256). This could result in a false sense of regulatory compliance and security. + +SFTP can be used to transfer files without requiring shell access to the user on the remote host (by using the [ForceCommand SSH config parameter](https://man7.org/linux/man-pages/man5/sshd_config.5.html)). In other words, a user on the remote host can be configured that has no shell access while still accepting file transfers. This can be relevant because limiting shell access is an important tool for securing a server. If a user has shell access, then if it is compromised, an attacker can run any permitted command for the user. By comparison, [SFTP has a narrower range of commands](https://man7.org/linux/man-pages/man1/sftp.1.html#INTERACTIVE_COMMANDS) that can be executed. + +SFTP can be further secured by creating an [SFTP *jail*](/docs/guides/limiting-access-with-sftp-jails-on-debian-and-ubuntu/), which limits which directory can be accessed on the remote server. + +### FTP/SFTP Clients + +One of the earliest data transfer mechanisms, FTP is a widely used protocol for transferring files over the internet. FTP clients like [Cyberduck](https://cyberduck.io/) and [FileZilla](https://filezilla-project.org/) are readily available for download; however, because FTP is an unencrypted protocol, its usage is now commonly discouraged for general purpose file transfers over the internet. SFTP is often recommended instead of FTP, with a variety of supporting SFTP clients (e.g., [WinSCP](https://winscp.net/eng/download.php) and [Tectia](https://www.ssh.com/products/tectia-ssh/)). Many legacy FTP clients now also support SFTP. As well, many of these clients also support transfers to and from object storage services, but they generally do not use the SFTP protocol to perform these transfers. + +#### Use Cases + +Most modern FTP/SFTP clients are capable of providing a unified GUI for accessing FTP, FTP over TLS (FTPS), and SFTP services. This allows users to copy files from local environments to remote servers with an interface that supports both legacy systems and newer machines running SFTP services. For users that require drag-and-drop functionality, FTP/SFTP clients enable them to transfer files remotely without having to use the command line. + +FTP is sparingly used on legacy systems and machines that are unable to support standard encryption protocols. Aside from these edge cases, FTP is commonly disabled on modern systems, with SFTP becoming the norm for transferring files securely over the internet. + +#### Pros and Cons + +FTP/SFTP clients are considered to have a low learning curve in comparison to other data transfer utilities. Users familiar with Windows File Explorer, macOS Finder, or File Manager on Linux desktops may find them intuitive and familiar due to their graphic interfaces. + +Not all FTP/SFTP clients share the same functionality, and some may lack certain features associated with other file transfer utilities. Users who require the ability to incorporate FTP/SFTP commands into scripts or automated jobs will be unable to do so with a GUI-based FTP/SFTP client, and are better suited choosing a command line-based utility.