Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update from upstream repo spf13/afero@master #18

Open
wants to merge 149 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
149 commits
Select commit Hold shift + click to select a range
b06a9e8
fix grammar in readme
rwaweber Feb 9, 2018
81cbd63
sftps: Fix test with latest API of the sftp package
xor-gate Mar 9, 2018
0408d60
Return closed error in mem.File.Write() after Close()
jslching May 17, 2018
d40851c
Add go.mod
bep Sep 7, 2018
709edc9
add ReadAt/Seek test
btc Nov 20, 2018
e151fe9
fix ReadAt
btc Nov 20, 2018
2236cc5
Fixed a small typo
yasoob Dec 4, 2018
5e9f8ec
Return os.ErrExist from CopyOnWriteFs.Mkdir/MkdirAll
bep Dec 16, 2018
a5d6946
Return nil in CopyOnWriteFs.MkdirAll when directory exists
bep Dec 16, 2018
617d93f
match: change strings.IndexAny to strings.ContainsAny
saromanov Jan 10, 2019
f4711e4
Fix missing bounds checking in UnionFile.Readdir
bep Jan 29, 2019
588a75e
Fix UnionFile.Readdir return when c <= 0
moorereason Mar 2, 2019
4e0df35
Add AIX support
Jul 4, 2019
338097a
Remove stale release notes from README
whereswaldon Nov 3, 2019
e016ffb
Add implementation for TempFile with wild card prefix
Jan 7, 2020
30c3c32
add chown method
bohdanlisovskyi Jan 22, 2020
ac701bd
Merge pull request #210 from Helflym/aix
0xmichalis Mar 27, 2020
00285cd
Merge pull request #155 from rwaweber/readme_grammar
0xmichalis Mar 27, 2020
de012fc
Merge pull request #187 from yasoob/patch-1
0xmichalis Mar 27, 2020
cc3719f
Merge pull request #182 from btc/btc/0002
0xmichalis Mar 27, 2020
491ff28
Merge pull request #223 from whereswaldon/patch-1
0xmichalis Mar 27, 2020
ff1b224
Merge pull request #157 from xor-gate/sftpfs-fix-test
0xmichalis Mar 27, 2020
6c364fc
Merge pull request #229 from ichekrygin/wild-temp-file
0xmichalis Mar 27, 2020
72d3215
Build and test CI improvements
0xmichalis Mar 27, 2020
f115557
Merge pull request #236 from kargakis/travis-update
0xmichalis Mar 28, 2020
4f00b06
Add zipfs, an archive/zip-based read-only filesystem
hillu Nov 15, 2017
d5bfeca
Work around root directory Open/Stat corner case leading to panic
hillu Mar 11, 2019
72cabd5
Eliminate another root-path-related corner case
hillu Mar 11, 2019
9b520d0
Use filepath.Separator in test code, to fix tests on Windows
hillu Mar 11, 2019
344ad9d
Don't hide errors while reading from ZIP files
hillu Mar 30, 2020
55428b4
Merge pull request #146 from hillu/zipfs
0xmichalis Apr 10, 2020
900cf72
Merge pull request #167 from jslching/master
0xmichalis Apr 10, 2020
d66a067
Add gofmt and go vet checks in CI
0xmichalis Apr 10, 2020
90f4ffe
Gofmt the project
0xmichalis Apr 10, 2020
2c061eb
Merge pull request #238 from kargakis/gofmt
0xmichalis Apr 10, 2020
02a77d4
Merge pull request #192 from saromanov/fix-match
0xmichalis Apr 10, 2020
ceb6a5e
Merge pull request #158 from kklin/sftp-openfile
0xmichalis Apr 10, 2020
5fd16ea
Fix zipfs.Readdir and zipfs.Readdirnames
cugu Apr 14, 2020
a2fefa4
Merge pull request #240 from cugu/patch-1
0xmichalis Apr 14, 2020
2f8d9cb
fix typo
dmitris Apr 17, 2020
70f98da
Add missing error check in RegexpFs Open
robinbraemer Apr 18, 2020
c17d293
Merge pull request #242 from robinbraemer/patch-1
0xmichalis Apr 21, 2020
7aa0da4
Change CRLF line endings to LF
ialidzhikov May 3, 2020
38a8ffc
Merge pull request #241 from dmitris/patch-1
0xmichalis May 4, 2020
c860826
Merge pull request #243 from ialidzhikov/nit/line-endings
0xmichalis May 4, 2020
529d861
Add an optional interface for Sylink and Readlink
Maldris Dec 10, 2019
819f7ad
Adding and optional Symlinker interface,
Maldris Dec 10, 2019
b4b149f
adding support for Linker and LinkReader
Maldris Dec 10, 2019
02af70d
adding tests for Linker and ReadLinker interfaces
Maldris Dec 11, 2019
a7dc6ae
Merge pull request #228 from Maldris/symlink
0xmichalis May 20, 2020
0b856b1
Add support for os.O_EXCL in afero.MemMapFs
campoy May 29, 2020
1524d0a
Merge pull request #245 from campoy/master
0xmichalis May 31, 2020
fc99a5d
Add failing test for reading a small zip file
JohnStarich Jun 26, 2020
e76d1f3
Fix panic when not filling up zipfs's read buffer
JohnStarich Jun 26, 2020
64b7ddd
Merge pull request #247 from JohnStarich/bugfix/zipfs-read-small-file
0xmichalis Jun 28, 2020
7b70cb1
Fix MemMapFs.Chmod changing non-perm bits
JohnStarich Jul 11, 2020
57ab25a
Use more appropriate func name for unrestricted mode changes
JohnStarich Jul 12, 2020
d443df9
Fix test for Windows
JohnStarich Jul 12, 2020
5d9e780
Merge pull request #249 from JohnStarich/bugfix/memfs-chmod-perm-only
0xmichalis Jul 13, 2020
b598fbb
Improve handling of mode bits
anishathalye Jul 15, 2020
238028b
Merge pull request #250 from anishathalye/improve-mode
0xmichalis Aug 2, 2020
7686d4f
go.mod: bump golang.org/x/text to v0.3.3
tklauser Aug 6, 2020
799ff74
Merge pull request #254 from tklauser/bump-x-text
0xmichalis Aug 9, 2020
0caf874
Fix typos in README code samples
cksriprajittichai Aug 12, 2020
6400260
Merge pull request #257 from cksriprajittichai/fix-typos-in-readme-code
0xmichalis Aug 12, 2020
066c8b0
Fix amount of files read in UnionFile.Readdir
std0 Aug 19, 2020
9e9dd69
mem: add test for Write/Seek
yerden Sep 1, 2020
d2521ec
mem: fix code style in Seek
yerden Sep 1, 2020
84f9c83
mem: fix Write setting wrong offset
yerden Sep 2, 2020
215b26b
Merge pull request #263 from yerden/fix-mmap-fs-seeker
0xmichalis Sep 2, 2020
27c9ee0
Merge pull request #260 from std0/union-readdir-fix
0xmichalis Sep 2, 2020
cb8b6bc
Make MemMapFs implement Lstater
LandonTClipp Sep 7, 2020
bf960e8
Removing use of testify
LandonTClipp Sep 13, 2020
277f220
Merge pull request #264 from LandonTClipp/memmapfs_lstatifpossible
0xmichalis Sep 14, 2020
a4ea980
Add tarfs implementation (reopen #265) (#266)
agimenez Sep 14, 2020
9fa7c3b
Do not panic on passing empty tar reader to tarfs
hasheddan Oct 5, 2020
efa411e
Merge pull request #268 from hasheddan/tarfs-panic
0xmichalis Oct 6, 2020
5dc7152
test
sreekanth370 Oct 20, 2020
6c06305
Merge pull request #271 from sreekanth370/master
0xmichalis Nov 7, 2020
d175ebc
mem.File.Truncate should use lock
rzajac Nov 13, 2020
736b98e
Merge pull request #273 from rzajac/mem-file-truncate-use-lock
0xmichalis Nov 15, 2020
cb1d580
Fix Chown() pull request errors
mfuterko Dec 7, 2020
5207eb9
Fix chown pull request errors
mfuterko Dec 7, 2020
cdee88c
Add go1.15 to the build matrix
mfuterko Dec 7, 2020
89dd0ad
Merge pull request #279 from mfuterko/implement-chown-functionality
0xmichalis Dec 7, 2020
816954d
Remove go1.13 from travis build matrix
0xmichalis Dec 7, 2020
f3723fe
Fix a typo
mfuterko Dec 8, 2020
32b5faa
Merge pull request #280 from mfuterko/master
0xmichalis Dec 9, 2020
7b47fea
fix(mem): do not overwrite old data in some cases (fixes #286)
tdakkota Feb 12, 2021
c597fe5
Add adapter to allow afero.Fs usage as io/fs.FS
xakep666 Mar 13, 2021
164e24d
Add reverse wrapper from io.FS to afero.Fs
xakep666 Mar 13, 2021
b9650fb
Merge pull request #291 from xakep666/feature/iofs
0xmichalis Mar 20, 2021
bc94f58
Merge pull request #287 from tdakkota/fix/issue-286
0xmichalis Mar 20, 2021
1a0f207
Add GCS Fs implementation
VOvchinnikov Mar 31, 2021
78acf3b
Changed abstraction to operate on client, not bucket
VOvchinnikov Apr 12, 2021
5dc5331
sftpfs Readdirnames,Readdir
lixiangzhong May 27, 2021
5b44be2
sftpfs OpenFile perm
lixiangzhong Jun 8, 2021
13ee679
Added test cases for `Mkdir` & `MkdirAll` to cover empty folder names
VOvchinnikov Aug 5, 2021
c122017
Added implementation to cover the new `Mkdir` & `MkdirAll` cases
VOvchinnikov Aug 5, 2021
090e652
read GOOGLE_APPLICATION_CREDENTIALS_JSON if it is set
dynamix Oct 5, 2021
d461d4f
go.mod: bump dependencies to latest version
tklauser Nov 10, 2021
f0ee686
Merge pull request #312 from tklauser/bump-deps
0xmichalis Nov 10, 2021
949437f
Readme: fix quotes of step 3
RaphaelPour Jul 9, 2021
061c13d
bugfix: RemoveAll
yearnfar Nov 14, 2021
21b977b
Merge branch 'master' into gcs
0xmichalis Dec 22, 2021
68b7156
Add README section for GCS Fs
0xmichalis Dec 22, 2021
d70f944
Merge pull request #331 from kargakis/gcs
0xmichalis Dec 22, 2021
039e386
Call `OpenFile` instead of `Open` in `CacheOnReadFs` (fixes #193)
pojntfx Dec 25, 2021
165e3dc
GCS fs: move all gcsfs related implementations to its own package
drakkan Dec 27, 2021
d8a4ef9
Merge pull request #333 from drakkan/gcsfs
0xmichalis Dec 28, 2021
4806203
Merge branch 'master' into master
lixiangzhong Jan 5, 2022
8298014
Merge pull request #306 from lixiangzhong/master
0xmichalis Jan 5, 2022
cd35605
Merge pull request #332 from pojntfx/master
0xmichalis Jan 5, 2022
c2c1d12
add modTime on folder creation
KastenMike Jan 10, 2022
9440921
fix actual value in test
KastenMike Jan 10, 2022
450b30f
Merge pull request #334 from KastenMike/add-modTime-to-folders
0xmichalis Feb 5, 2022
e08384e
return no error if file doesn't exist
typerat Feb 22, 2022
25e8a0a
fix typo
typerat Feb 22, 2022
d6369bb
add test for RemoveAll
typerat Feb 22, 2022
b25fb10
clarify error message
typerat Mar 10, 2022
931302a
Merge pull request #340 from typerat/fix-gcsfs-removeall-notfound
mbertschler Mar 11, 2022
a2bf4c8
Merge pull request #329 from yearnfar/master
mbertschler Mar 11, 2022
8d9b795
go mod tidy
mbertschler Mar 11, 2022
c45d7c7
Bump minimum `go` version to `1.16`
jeffwidman Mar 16, 2022
100c9a6
Merge pull request #346 from jeffwidman/bump-minimum-go-version
0xmichalis Apr 1, 2022
015be45
Move CI tests to GitHub Action
bep Jul 14, 2022
9439436
all: Run gofmt -s -w
bep Jul 14, 2022
939bf3d
Fix test failures on Windows
bep Jul 14, 2022
52b6417
Fix staticcheck lint errors
bep Jul 14, 2022
c92ae36
Make IOFS.ReadDir check for fs.ReadDirFile
bep Jul 14, 2022
b0a534a
Update README.md
bep Jul 14, 2022
0aa65ed
Fix sorting in IOFS.ReadDir
bep Jul 19, 2022
2a70f2b
Make mem.File implement fs.ReadDirFile
bep Jul 15, 2022
a800a9d
Fix concurrency issue in MemMapFs.Mkdir/MkdirAll
bep Nov 14, 2022
3b1116b
memmap: Fix renaming a dir with sub-directories (#239)
nono Feb 23, 2023
a6023d2
github: Update Go build matrix
bep Feb 23, 2023
cf95922
all: Run gofumpt -l -w .
bep Feb 23, 2023
4ba522f
fix(deps): golang.org/x/crypto, golang.org/x/text
gnanderson Aug 11, 2022
1edd01a
Run go mod tidy
bep Mar 6, 2023
501e8d3
Fix io.Seek* deprecation errors
bep Mar 6, 2023
d92c300
Fix failing tests on Windows with Go 1.20
bep Mar 6, 2023
1882278
Add sftpfs files ReadAt method
MRtecno98 Aug 30, 2022
45ef346
Add zos build flags
msradam Jan 30, 2023
ee6eef7
Fix rename mem fs with descendants
hanagantig Sep 22, 2023
c419971
github: Update Go versions
bep Nov 28, 2023
2a63c9c
deps: Update all direct dependencies
bep Nov 28, 2023
2fd19a1
github: Add permissions to workflow
bep Nov 28, 2023
eb6bef0
github: Format workflow file
bep Nov 28, 2023
5c4385a
github: Bump GitHub actions versions
bep Nov 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
on:
push:
branches: [main]
pull_request:
name: Test
permissions:
contents: read
jobs:
test:
strategy:
matrix:
go-version: [1.19.x, 1.20.x, 1.21.x]
platform: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.platform }}
steps:
- name: Install Go
uses: actions/setup-go@v4
with:
go-version: ${{ matrix.go-version }}
- name: Install staticcheck
if: matrix.go-version == '1.21.x'
run: go install honnef.co/go/tools/cmd/staticcheck@latest
shell: bash
- name: Update PATH
run: echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
shell: bash
- name: Checkout code
uses: actions/checkout@v3
- name: Fmt
if: matrix.platform != 'windows-latest' # :(
run: "diff <(gofmt -d .) <(printf '')"
shell: bash
- name: Vet
run: go vet ./...
- name: Staticcheck
if: matrix.go-version == '1.21.x'
run: staticcheck ./...
- name: Test
run: go test -race ./...
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
sftpfs/file1
sftpfs/test/
21 changes: 0 additions & 21 deletions .travis.yml

This file was deleted.

60 changes: 25 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

A FileSystem Abstraction System for Go

[![Build Status](https://travis-ci.org/spf13/afero.svg)](https://travis-ci.org/spf13/afero) [![Build status](https://ci.appveyor.com/api/projects/status/github/spf13/afero?branch=master&svg=true)](https://ci.appveyor.com/project/spf13/afero) [![GoDoc](https://godoc.org/github.com/spf13/afero?status.svg)](https://godoc.org/github.com/spf13/afero) [![Join the chat at https://gitter.im/spf13/afero](https://badges.gitter.im/Dev%20Chat.svg)](https://gitter.im/spf13/afero?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![Test](https://github.com/spf13/afero/actions/workflows/test.yml/badge.svg)](https://github.com/spf13/afero/actions/workflows/test.yml) [![GoDoc](https://godoc.org/github.com/spf13/afero?status.svg)](https://godoc.org/github.com/spf13/afero) [![Join the chat at https://gitter.im/spf13/afero](https://badges.gitter.im/Dev%20Chat.svg)](https://gitter.im/spf13/afero?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

# Overview

Afero is an filesystem framework providing a simple, uniform and universal API
Afero is a filesystem framework providing a simple, uniform and universal API
interacting with any filesystem, as an abstraction layer providing interfaces,
types and methods. Afero has an exceptionally clean interface and simple design
without needless constructors or initialization methods.
Expand All @@ -18,7 +18,7 @@ and benefit of the os and ioutil packages.
Afero provides significant improvements over using the os package alone, most
notably the ability to create mock and testing filesystems without relying on the disk.

It is suitable for use in a any situation where you would consider using the OS
It is suitable for use in any situation where you would consider using the OS
package as it provides an additional abstraction that makes it easy to use a
memory backed file system during testing. It also adds support for the http
filesystem for full interoperability.
Expand All @@ -33,16 +33,16 @@ filesystem for full interoperability.
* Support for compositional (union) file systems by combining multiple file systems acting as one
* Specialized backends which modify existing filesystems (Read Only, Regexp filtered)
* A set of utility functions ported from io, ioutil & hugo to be afero aware

* Wrapper for go 1.16 filesystem abstraction `io/fs.FS`

# Using Afero

Afero is easy to use and easier to adopt.

A few different ways you could use Afero:

* Use the interfaces alone to define you own file system.
* Wrap for the OS packages.
* Use the interfaces alone to define your own file system.
* Wrapper for the OS packages.
* Define different filesystems for different parts of your application.
* Use Afero for mock filesystems while testing

Expand Down Expand Up @@ -79,11 +79,11 @@ would.

So if my application before had:
```go
os.Open('/tmp/foo')
os.Open("/tmp/foo")
```
We would replace it with:
```go
AppFs.Open('/tmp/foo')
AppFs.Open("/tmp/foo")
```

`AppFs` being the variable we defined above.
Expand All @@ -94,6 +94,7 @@ AppFs.Open('/tmp/foo')
File System Methods Available:
```go
Chmod(name string, mode os.FileMode) : error
Chown(name string, uid, gid int) : error
Chtimes(name string, atime time.Time, mtime time.Time) : error
Create(name string) : File, error
Mkdir(name string, perm os.FileMode) : error
Expand Down Expand Up @@ -227,7 +228,7 @@ operation and a mock filesystem during testing or as needed.

```go
appfs := afero.NewOsFs()
appfs.MkdirAll("src/a", 0755))
appfs.MkdirAll("src/a", 0755)
```

## Memory Backed Storage
Expand All @@ -241,7 +242,7 @@ safely.

```go
mm := afero.NewMemMapFs()
mm.MkdirAll("src/a", 0755))
mm.MkdirAll("src/a", 0755)
```

#### InMemoryFile
Expand All @@ -258,6 +259,18 @@ system using InMemoryFile.
Afero has experimental support for secure file transfer protocol (sftp). Which can
be used to perform file operations over a encrypted channel.

### GCSFs

Afero has experimental support for Google Cloud Storage (GCS). You can either set the
`GOOGLE_APPLICATION_CREDENTIALS_JSON` env variable to your JSON credentials or use `opts` in
`NewGcsFS` to configure access to your GCS bucket.

Some known limitations of the existing implementation:
* No Chmod support - The GCS ACL could probably be mapped to *nix style permissions but that would add another level of complexity and is ignored in this version.
* No Chtimes support - Could be simulated with attributes (gcs a/m-times are set implicitly) but that's is left for another version.
* Not thread safe - Also assumes all file operations are done through the same instance of the GcsFs. File operations between different GcsFs instances are not guaranteed to be consistent.


## Filtering Backends

### BasePathFs
Expand Down Expand Up @@ -306,7 +319,7 @@ Any Afero FileSystem can be used as an httpFs.

```go
httpFs := afero.NewHttpFs(<ExistingFS>)
fileserver := http.FileServer(httpFs.Dir(<PATH>)))
fileserver := http.FileServer(httpFs.Dir(<PATH>))
http.Handle("/", fileserver)
```

Expand Down Expand Up @@ -380,8 +393,6 @@ The following is a short list of possible backends we hope someone will
implement:

* SSH
* ZIP
* TAR
* S3

# About the project
Expand All @@ -406,28 +417,7 @@ Googles very well.

## Release Notes

* **0.10.0** 2015.12.10
* Full compatibility with Windows
* Introduction of afero utilities
* Test suite rewritten to work cross platform
* Normalize paths for MemMapFs
* Adding Sync to the file interface
* **Breaking Change** Walk and ReadDir have changed parameter order
* Moving types used by MemMapFs to a subpackage
* General bugfixes and improvements
* **0.9.0** 2015.11.05
* New Walk function similar to filepath.Walk
* MemMapFs.OpenFile handles O_CREATE, O_APPEND, O_TRUNC
* MemMapFs.Remove now really deletes the file
* InMemoryFile.Readdir and Readdirnames work correctly
* InMemoryFile functions lock it for concurrent access
* Test suite improvements
* **0.8.0** 2014.10.28
* First public version
* Interfaces feel ready for people to build using
* Interfaces satisfy all known uses
* MemMapFs passes the majority of the OS test suite
* OsFs passes the majority of the OS test suite
See the [Releases Page](https://github.com/spf13/afero/releases).

## Contributing

Expand Down
11 changes: 7 additions & 4 deletions afero.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,17 +91,20 @@ type Fs interface {
// The name of this FileSystem
Name() string

//Chmod changes the mode of the named file to mode.
// Chmod changes the mode of the named file to mode.
Chmod(name string, mode os.FileMode) error

//Chtimes changes the access and modification times of the named file
// Chown changes the uid and gid of the named file.
Chown(name string, uid, gid int) error

// Chtimes changes the access and modification times of the named file
Chtimes(name string, atime time.Time, mtime time.Time) error
}

var (
ErrFileClosed = errors.New("File is closed")
ErrOutOfRange = errors.New("Out of range")
ErrTooLarge = errors.New("Too large")
ErrOutOfRange = errors.New("out of range")
ErrTooLarge = errors.New("too large")
ErrFileNotFound = os.ErrNotExist
ErrFileExists = os.ErrExist
ErrDestinationExists = os.ErrExist
Expand Down
Loading