• Skip to main content
  • Skip to primary sidebar
…

...

cuộc sống là phải luôn khám phá.

  • Home
  • Music Box
  • Các trang web hay
  • Giới thiệu
  • Liên kết website
Bạn đang ở đây:Trang chủ / testing / (Rclone) Tips to use Rclone

(Rclone) Tips to use Rclone

25/10/2021 tác giả: Duy Nghĩa Bình luận >

RClone_logo

Rclone

Rclone is a command line program to manage files on cloud storage. It is a feature rich alternative to cloud vendors’ web storage interfaces. Over 40 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols.

Rclone has powerful cloud equivalents to the unix commands rsync, cp, mv, mount, ls, ncdu, tree, rm, and cat. Rclone’s familiar syntax includes shell pipeline support, and  –dry-run protection. It is used at the command line, in scripts or via its API.

Rclone documentation and in-depth guide on how to install can be found at rclone.org.

Rclone helps you:

  • Backup (and encrypt) files to cloud storage
  • Restore (and decrypt) files from cloud storage
  • Mirror cloud data to other cloud services or locally
  • Migrate data to cloud, or between cloud storage vendors
  • Mount multiple, encrypted, cached or diverse cloud storage as a disk
  • Analyse and account for data held on cloud storage using lsf, ljson, size, ncdu
  • Union file systems together to present multiple local and/or cloud file systems as one.
Warning: I haven’t verified all below commands and parameters below as this post, for now, is intended for personal reference only. I will come back here and make a note about how every commands together with their flags work. You should read very carefully at the provided sources before attempting to implement on your real data, because it could potentially cause data loss. Remember to always back up your critical data before trying new things.

Fixing Duplicates in Google Drive using Rclone to Dedupe:

<#
RCLONE HELP
    https://rclone.org/commands/rclone_dedupe/
Useful options I choose
    --max-depth int
    --dry-run
    --log-file=PATH
    --tpslimit 1 # can help prevent rate limiting errors you might see if you run verbose x2, ie `-vv`
    --checkers 1
#>

# Install Rclone and a log viewer to make it nice to review results
choco upgrade rclone -y
choco upgrade tailblazer -y # nice tail log view for streaming rclone activity

# Alias for usage
new-alias rclone -value "C:\Program Files\rclone\rclone-v1.42-windows-amd64\rclone.exe" -force

# logging to this directory
New-Item 'C:\temp' -ItemType Directory -force
$LogFile = "C:\temp\rclone-$(Get-Date -format 'yyyy-MM-dd').log"

# Tested Against Folder With Dups With Dry Run
rclone dedupe newest googleappsdrive:Test --log-file=$LogFile --dry-run

# Ran against folder and it removed 3 of the 4, leaving only one file, now deduplicated
rclone dedupe newest googleappsdrive:Test --log-file=$LogFile

# another folder with dups but larger. This ran into issues when I didn't limit depth
rclone dedupe newest --dry-run googleappsdrive:"Amazon Drive\Development" --log-file=$LogFile --max-depth 2

# merge the root folder level only of duplicate folders
rclone dedupe newest googleappsdrive:"" --drive-skip-gdocs --log-file=$LogFile -vv --tpslimit 4 --transfers 1 --fast-list --max-depth 1 --stats=30s

# now dig into my Lightroom Library and deduplicate a bit more. I did this to confirm it was working before I did everything in my folders.
rclone dedupe newest googleappsdrive:"Lightroom" --drive-skip-gdocs --log-file=$LogFile -vv --tpslimit 4 --transfers 1 --fast-list --max-depth 1 --stats=30s
rclone dedupe newest googleappsdrive:"Lightroom" --drive-skip-gdocs --log-file=$LogFile -vv --tpslimit 4 --transfers 1 --fast-list --max-depth 2 --stats=30s
rclone dedupe newest googleappsdrive:"Lightroom" --drive-skip-gdocs --log-file=$LogFile -vv --tpslimit 4 --transfers 1 --fast-list --max-depth 3 --stats=30s

# now that I've got everything figured out, I ran against the entire drive.
rclone dedupe newest googleappsdrive:"" --drive-skip-gdocs --log-file=$LogFile -vv --tpslimit 4 --transfers 1 --fast-list --stats=30s

Use RClone for cloud Storage and coordination

Rclone can connect your cloud storage (GDrive, Dropbox, OneDrive, Nextcloud … ) to your PC like a HDD drive, a good alternative to Google Drive File Stream.

rclone mount --vfs-cache-mode full Gdrive: z: -v  
rclone mount --vfs-cache-mode full Gdrive: x: -v 

How to transfer files that marked as abused

--drive-acknowledge-abuse     # Set to allow files which return cannotDownloadAbusiveFile to be downloaded.
rclone sync -P A: B: --drive-acknowledge-abuse --exclude "/some_folder/**"

Reference links

For global flags, check the docs.

https://www.osc.edu/resources/getting_started/howto/howto_use_rclone_to_upload_data

….

(Related posts:)

https://pquan.info/cach-thiet-lap-de-cho-phep-ghi-ext4-ubuntu-linux-mint/

Reader Interactions

Để lại bình luận: Hủy

Email của bạn sẽ không được hiển thị công khai. Các trường bắt buộc được đánh dấu *

Primary Sidebar

Chuyên mục

  • fun
  • IT
  • News
  • testing

python

Python Basic: Getting to Python (part 4)

In short, Lists are used to store multiple items in a single variable, they are convenient data structures for representing a sequence of data. In that regard, a list is similar to a string, except a string can hold only characters, however, a list can hold any Python object.

python

Python Basic: Getting to Python (part 3)

Python Functions: (user-defined or from libraries) function is a block of code which only runs when it is called, to do a complete job (module), is named, and can be called to execute multiple times at many places in the program. What happens if you don’t use function? What I can think about is repetitive works, hard to debug, hard to expand the whole code, so usually when working on a large project you need to split your code into many small modules.

Tạo một website dễ dàng với Hugo

Hôm nay chúng ta cùng thử generate 1 website với Hugo, theo như lời giới thiệu ở trang chủ “The world’s fastest framework for building websites.”

Steam_logo

DMCA.com Protection Status

Copyright ©2022 · pquan.info - All Rights Reserved ·