- Author
- Devresse Adrien ( adrie.nosp@m.n.de.nosp@m.vress.nosp@m.e@ce.nosp@m.rn.ch )
From CERN IT-GT-DMS ( lcgut.nosp@m.il-s.nosp@m.uppor.nosp@m.t@ce.nosp@m.rn.ch ) mailing list : davix.nosp@m.-dev.nosp@m.el@ce.nosp@m.rn.c.nosp@m.h
DAVIX
Davix is a lightweight toolkit for remote file interactions with HTTP based protocols.
Davix aims to supports all the needed features for an efficient file access and file management in grid and clouds : SSL Session reuse, X509 client auth, VOMS credential, S3 auth, Vector operations (Partial reads, multi-range, single range), Partial PUT / PATCH, Fail-over, Multi-streams ( Metalinks), Redirection support for all operations, Redirections caching, Webdav parsing, Right Management (ACL), Meta-data functions ( mkdir, rmdir, unlink, etc.. ), Chunked transfert, etc....
Davix supports the protocols
The Davix philosophy can be summarized as
- Just Access Files, don't loose time with the protocol tricks
- Keep It Simple Stupid
- Efficient
- Portable
DAVIX API :
File API : Davix::DavFile
Posix-like API : Davix::DavPosix
Entry point API : davix.hpp
Davix is yet an other libcurl ?
In short : No
Libcurl defines itself as a "client side URL transfer".
it provides "protocol level" API, you compose your http queries mannually.
- Davix offers a "file level" API.
With Davix, you access and manage your data and do not have to know anything about Http and how to tune queries. Davix tends to be of one level higher and provides a complete API for remote I/O and remote file management.
Examples :
File Usage
Create a directory :
DavixError* tmp_err=NULL;
DavFile f(context, url);
p.makeCollection(NULL, &tmp_err);
Get a full file content:
DavixError* tmp_err=NULL;
DavFile f(context, "http://mysite.org/file");
int fd = open("/tmp/local_file", O_WRONLY | O_CREAT);
if( p.getToFd(NULL,fd, &tmp_err) < 0)
std::cerr << "Error: " << tmp_err->getErrMsg() << std::endl;
Execute a partial GET :
char buffer[255] = {0}
DavixError* tmp_err=NULL;
DavFile f(context, "http://mysite.org/file");
if( p.readPartial(NULL, buffer, 100, 200
&tmp_err) <0 )
std::cerr << "Error: " << tmp_err->getErrMsg() << std::endl;
else
std::cout << "Content: " << buffer << std::endl;
Execute a Vector Operation :
char buffer[255] = {0}
DavixError* tmp_err=NULL;
DavFile f(context, "http://mysite.org/file");
DavIOVecInput in[3];
DavIOVecOutput ou[3];
if( p.readPartial(NULL, buffer, 100, 200
&tmp_err) <0 )
std::cerr << "Error: " << tmp_err->getErrMsg() << std::endl;
else
std::cout << "Content: " << buffer << std::endl;
POSIX Usage
Stat query :
p.
stat(
"https://mywebdav-server.org/mydir/", &stat, &tmp_err);
random I/O :
fd= p.
open(NULL,
"https://mywebdav-server.org/myfile.jpg", O_RDONLY, &tmp_err);
p.
read(fd, buffer, size, &tmp_err);
p.
pread(fd, buffer, size2, offset, &tmp_err);
LOW LEVEL Usage
req.addHeaderField(...)
req.setRequestMethod("PUT")
req.executeRequest(...);
How to compile :
- Davix Dependencies :
- openssl
- libxml-2.0
- Doxygen ( optional, for documentation generation )
- Davix Portability :
- Should run on Windows and any POSIX compatible Operating system
- Any contribution to support a new plateform is welcome
- Tested on Debian, Ubuntu, Fedora 18/19, Scientific Linux 5/6
- Tested on Windows under MinGW
- Tested with clang and GCC.
- Compile :
- " 1. git clone http://git.cern.ch/pub/davix "
- " 2. cd davix "
- " 3. mkdir build; cd build"
- " 4. cmake ../"
- " 5. make "
- Generate doc :
- Compile and run unit tests :
- cmake -DUNIT_TESTS=TRUE ../
- make
- make test
- Compile & execute func tests : warning : functionals test needs davserver and a valid credential
- " 4. cmake -DFUNCTIONAL_TESTS=TRUE ../ "
- " 5. . ../test/setup_test_env.sh
- " 5. make; make test"
- make RPMS :
- ./packaging/bin/packager_rpm.sh ./packaging/rpm/specs/ ./
- mock -r [mycfg] RPMS/davix-[...].src.rpm
Play with davix command line tool :
davix has a set of command line tools for testing purpose and demonstration
-> davix-ls: file listing
-> davix-get: download operations
-> davix-put: upload operations
-> davix: low level query composition
TODO in Davix :
- MacOSX portability check
- Kerberos support
- Metalink support
- map S3 bucket operations
- ACL support
- (?) CDMI support
please contact us on davix-devel@cern.ch ( CERN e-group & mailing list ) or on adrien.devresse@cern.ch
Any contribution is welcome
Davix Website :
https://svnweb.cern.ch/trac/lcgutil/wiki/davix