XRootD
XrdPfc::IOFile Class Reference

Downloads original file into a single file on local disk. Handles read requests as they come along. More...

#include <XrdPfcIOFile.hh>

+ Inheritance diagram for XrdPfc::IOFile:
+ Collaboration diagram for XrdPfc::IOFile:

Public Member Functions

 IOFile (XrdOucCacheIO *io, Cache &cache)
 
 ~IOFile ()
 
void DetachFinalize () override
 Abstract virtual method of XrdPfc::IO Called to destruct the IO object after it is no longer used. More...
 
long long FSize () override
 
int Fstat (struct stat &sbuff) override
 
bool HasFile () const
 Check if File was opened successfully. More...
 
bool ioActive () override
 Abstract virtual method of XrdPfc::IO Called to check if destruction needs to be done in a separate task. More...
 
virtual int pgRead (char *buff, long long offs, int rdlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
void pgRead (XrdOucCacheIOCB &iocb, char *buff, long long off, int size, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0) override
 
virtual void pgRead (XrdOucCacheIOCB &iocb, char *buff, long long offs, int rdlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
int Read (char *buff, long long off, int size) override
 Pass Read request to the corresponding File object. More...
 
void Read (XrdOucCacheIOCB &iocb, char *buff, long long off, int size) override
 
int ReadV (const XrdOucIOVec *readV, int n) override
 Pass ReadV request to the corresponding File object. More...
 
void ReadV (XrdOucCacheIOCB &iocb, const XrdOucIOVec *readV, int n) override
 
void Update (XrdOucCacheIO &iocp) override
 
- Public Member Functions inherited from XrdPfc::IO
 IO (XrdOucCacheIO *io, Cache &cache)
 
virtual XrdOucCacheIOBase ()
 Original data source. More...
 
bool Detach (XrdOucCacheIOCD &iocdP) final
 
XrdOucCacheIOGetInput ()
 
const char * GetLocation ()
 
XrdSysTraceGetTrace ()
 
const char * Path () override
 Original data source URL. More...
 
int Sync () override
 
virtual int Sync ()=0
 
virtual void Sync (XrdOucCacheIOCB &iocb)
 
virtual int Trunc (long long offs)=0
 
int Trunc (long long Offset) override
 
virtual void Trunc (XrdOucCacheIOCB &iocb, long long offs)
 
virtual int Write (char *buff, long long offs, int wlen)=0
 
int Write (char *Buffer, long long Offset, int Length) override
 
virtual void Write (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wlen)
 
- Public Member Functions inherited from XrdOucCacheIO
 XrdOucCacheIO ()
 Construct and Destructor. More...
 
virtual const char * Location (bool refresh=false)
 
virtual int pgRead (char *buff, long long offs, int rdlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
virtual int pgWrite (char *buff, long long offs, int wrlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
virtual void pgWrite (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wrlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
 
virtual void Preread (aprParms &Parms)
 
virtual void Preread (long long offs, int rlen, int opts=0)
 
virtual void Sync (XrdOucCacheIOCB &iocb)
 
virtual void Trunc (XrdOucCacheIOCB &iocb, long long offs)
 
virtual void Write (XrdOucCacheIOCB &iocb, char *buff, long long offs, int wlen)
 
virtual int WriteV (const XrdOucIOVec *writV, int wnum)
 
virtual void WriteV (XrdOucCacheIOCB &iocb, const XrdOucIOVec *writV, int wnum)
 

Additional Inherited Members

- Static Public Attributes inherited from XrdOucCacheIO
static const uint64_t forceCS = 0x0000000000000001ULL
 
static const int SingleUse = 0x0001
 Mark pages for single use. More...
 
- Protected Member Functions inherited from XrdPfc::IO
std::string GetFilename ()
 
const char * GetPath ()
 
unsigned short ObtainReadSid ()
 
const char * RefreshLocation ()
 
- Protected Member Functions inherited from XrdOucCacheIO
virtual ~XrdOucCacheIO ()
 
- Protected Attributes inherited from XrdPfc::IO
RAtomic_int m_active_read_reqs
 number of active read requests More...
 
Cachem_cache
 reference to Cache object More...
 
const char * m_traceID
 

Detailed Description

Downloads original file into a single file on local disk. Handles read requests as they come along.

Definition at line 39 of file XrdPfcIOFile.hh.

Constructor & Destructor Documentation

◆ IOFile()

IOFile::IOFile ( XrdOucCacheIO io,
Cache cache 
)

Definition at line 36 of file XrdPfcIOFile.cc.

36  :
37  IO(io, cache),
38  m_file(0),
39  m_localStat(0)
40 {
41  m_file = Cache::GetInstance().GetFile(GetFilename(), this);
42 }
File * GetFile(const std::string &, IO *, long long off=0, long long filesize=0)
Definition: XrdPfc.cc:412
static Cache & GetInstance()
Singleton access.
Definition: XrdPfc.cc:160
std::string GetFilename()
Definition: XrdPfcIO.hh:56
IO(XrdOucCacheIO *io, Cache &cache)
Definition: XrdPfcIO.cc:6

References XrdPfc::Cache::GetFile(), XrdPfc::IO::GetFilename(), and XrdPfc::Cache::GetInstance().

+ Here is the call graph for this function:

◆ ~IOFile()

IOFile::~IOFile ( )

Definition at line 45 of file XrdPfcIOFile.cc.

46 {
47  // called from Detach() if no sync is needed or
48  // from Cache's sync thread
49  TRACEIO(Debug, "~IOFile() " << this);
50 
51  delete m_localStat;
52 }
#define TRACEIO(act, x)
Definition: XrdPfcTrace.hh:59

References Macaroons::Debug, and TRACEIO.

Member Function Documentation

◆ DetachFinalize()

void IOFile::DetachFinalize ( )
overridevirtual

Abstract virtual method of XrdPfc::IO Called to destruct the IO object after it is no longer used.

Implements XrdPfc::IO.

Definition at line 160 of file XrdPfcIOFile.cc.

161 {
162  // Effectively a destructor.
163 
164  TRACE(Info, "DetachFinalize() " << this);
165 
166  m_file->RequestSyncOfDetachStats();
167  Cache::GetInstance().ReleaseFile(m_file, this);
168 
169  delete this;
170 }
#define TRACE(act, x)
Definition: XrdTrace.hh:63
void ReleaseFile(File *, IO *)
Definition: XrdPfc.cc:493
void RequestSyncOfDetachStats()
Flags that detach stats should be written out in final sync. Called from CacheIO upon Detach.
Definition: XrdPfcFile.cc:265
Status of cached file. Can be read from and written into a binary file.
Definition: XrdPfcInfo.hh:45

References XrdPfc::Cache::GetInstance(), XrdPfc::Cache::ReleaseFile(), XrdPfc::File::RequestSyncOfDetachStats(), and TRACE.

+ Here is the call graph for this function:

◆ FSize()

long long IOFile::FSize ( )
overridevirtual

Obtain size of the file.

Returns
Size of the file in bytes.

Implements XrdOucCacheIO.

Definition at line 69 of file XrdPfcIOFile.cc.

70 {
71  return m_file->GetFileSize();
72 }
long long GetFileSize()
Definition: XrdPfcFile.hh:279

References XrdPfc::File::GetFileSize().

+ Here is the call graph for this function:

◆ Fstat()

int IOFile::Fstat ( struct stat sbuff)
overridevirtual

Perform an fstat() operation (defaults to passthrough).

Parameters
sbuffreference to the stat buffer to be filled in. Only fields st_size, st_blocks, st_mtime (st_atime and st_ctime may be set to st_mtime), st_ino, and st_mode need to be set. All other fields are preset and should not be changed.
Returns
<0 - fstat failed, value is -errno. =0 - fstat succeeded, sbuff holds stat information. >0 - fstat could not be done, forward operation to next level.

Reimplemented from XrdOucCacheIO.

Definition at line 55 of file XrdPfcIOFile.cc.

56 {
57  int res = 0;
58  if( ! m_localStat)
59  {
60  res = initCachedStat();
61  if (res) return res;
62  }
63 
64  memcpy(&sbuff, m_localStat, sizeof(struct stat));
65  return 0;
66 }
int stat(const char *path, struct stat *buf)

References stat().

+ Here is the call graph for this function:

◆ HasFile()

bool XrdPfc::IOFile::HasFile ( ) const
inline

Check if File was opened successfully.

Definition at line 49 of file XrdPfcIOFile.hh.

49 { return m_file != 0; }

Referenced by XrdPfc::Cache::Attach().

+ Here is the caller graph for this function:

◆ ioActive()

bool IOFile::ioActive ( )
overridevirtual

Abstract virtual method of XrdPfc::IO Called to check if destruction needs to be done in a separate task.

Implements XrdPfc::IO.

Definition at line 153 of file XrdPfcIOFile.cc.

154 {
155  RefreshLocation();
156  return m_file->ioActive(this);
157 }
bool ioActive(IO *io)
Initiate close. Return true if still IO active. Used in XrdPosixXrootd::Close()
Definition: XrdPfcFile.cc:188
const char * RefreshLocation()
Definition: XrdPfcIO.hh:57

References XrdPfc::File::ioActive(), and XrdPfc::IO::RefreshLocation().

+ Here is the call graph for this function:

◆ pgRead() [1/3]

int XrdOucCacheIO::pgRead

Definition at line 190 of file XrdOucCache.cc.

45 {
46  (void)csfix;
47  int bytes;
48 
49 // Read the data into the buffer
50 //
51  bytes = Read(buff, offs, rdlen);
52 
53 // Calculate checksums if so wanted
54 //
55  if (bytes > 0 && (opts & forceCS))
56  XrdOucPgrwUtils::csCalc((const char *)buff, (ssize_t)offs,
57  (size_t)bytes, csvec);
58 
59 // All done
60 //
61  return bytes;
62 }
struct myOpts opts
static const uint64_t forceCS
Definition: XrdOucCache.hh:188
static void csCalc(const char *data, off_t offs, size_t count, uint32_t *csval)
int Read(char *buff, long long off, int size) override
Pass Read request to the corresponding File object.

◆ pgRead() [2/3]

void IOFile::pgRead ( XrdOucCacheIOCB iocb,
char *  buff,
long long  offs,
int  rdlen,
std::vector< uint32_t > &  csvec,
uint64_t  opts = 0,
int *  csfix = 0 
)
overridevirtual

Read file pages and checksums using asynchronous I/O (default sync).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buffpointer to buffer where the bytes are to be placed.
offsThe offset where the read is to start.
rdlenThe number of bytes to read.
csvecA vector which will be filled with the corresponding CRC32C checksum for each page or page segment.
optsProcessing options: forceCS - always return checksums even when not available.
csfixWhen not nil, returns the number of corrected checksum errs.

Reimplemented from XrdOucCacheIO.

Definition at line 225 of file XrdPfcIOFile.cc.

227 {
228  struct ZHandler : ReadReqRH
229  { using ReadReqRH::ReadReqRH;
230  IOFile *m_io = nullptr;
231  std::function<void (int)> m_lambda {0};
232 
233  void Done(int result) override {
234  if (m_lambda) m_lambda(result);
235  m_io->ReadEnd(result, this);
236  }
237  };
238 
240 
241  auto *rh = new ZHandler(ObtainReadSid(), &iocb);
242  rh->m_io = this;
243 
244  TRACEIO(Dump, "pgRead() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
245 
247  rh->m_lambda = [=, &csvec](int result) {
248  if (result > 0)
249  XrdOucPgrwUtils::csCalc((const char *)buff, (ssize_t)off, (size_t)result, csvec);
250  };
251 
252  int retval = ReadBegin(buff, off, size, rh);
253  if (retval != -EWOULDBLOCK)
254  {
255  rh->Done(retval);
256  }
257 }
Downloads original file into a single file on local disk. Handles read requests as they come along.
Definition: XrdPfcIOFile.hh:40
RAtomic_int m_active_read_reqs
number of active read requests
Definition: XrdPfcIO.hh:72
unsigned short ObtainReadSid()
Definition: XrdPfcIO.hh:59
@ hex1
Definition: XrdSysTrace.hh:42
ReadReqRH(unsigned short sid, XrdOucCacheIOCB *iocb)
Definition: XrdPfcFile.hh:67

References XrdPfc::ReadReqRH::ReadReqRH(), XrdOucPgrwUtils::csCalc(), XrdOucCacheIO::forceCS, Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), opts, and TRACEIO.

+ Here is the call graph for this function:

◆ pgRead() [3/3]

virtual void XrdOucCacheIO::pgRead
inline

Read file pages and checksums using asynchronous I/O (default sync).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buffpointer to buffer where the bytes are to be placed.
offsThe offset where the read is to start.
rdlenThe number of bytes to read.
csvecA vector which will be filled with the corresponding CRC32C checksum for each page or page segment.
optsProcessing options: forceCS - always return checksums even when not available.
csfixWhen not nil, returns the number of corrected checksum errs.

Definition at line 214 of file XrdOucCache.hh.

221  {iocb.Done(pgRead(buff, offs, rdlen, csvec, opts, csfix));}
virtual int pgRead(char *buff, long long offs, int rdlen, std::vector< uint32_t > &csvec, uint64_t opts=0, int *csfix=0)
Definition: XrdOucCache.cc:39

◆ Read() [1/2]

int IOFile::Read ( char *  buff,
long long  off,
int  size 
)
overridevirtual

Pass Read request to the corresponding File object.

Implements XrdOucCacheIO.

Definition at line 178 of file XrdPfcIOFile.cc.

179 {
181 
182  auto *rh = new ReadReqRHCond(ObtainReadSid(), nullptr);
183 
184  TRACEIO(Dump, "Read() sync " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
185 
186  rh->m_cond.Lock();
187  int retval = ReadBegin(buff, off, size, rh);
188  if (retval == -EWOULDBLOCK)
189  {
190  rh->m_cond.Wait();
191  retval = rh->m_retval;
192  }
193  rh->m_cond.UnLock();
194 
195  return ReadEnd(retval, rh);
196 }

References Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ Read() [2/2]

void IOFile::Read ( XrdOucCacheIOCB iocb,
char *  buff,
long long  offs,
int  rlen 
)
overridevirtual

Perform an asynchronous read (defaults to synchronous).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
buffpointer to the buffer to receive the results. The buffer must remain valid until the callback is invoked.
offsthe offset into the file.
rlenthe number of bytes to read.

Reimplemented from XrdOucCacheIO.

Definition at line 199 of file XrdPfcIOFile.cc.

200 {
201  struct ZHandler : ReadReqRH
202  { using ReadReqRH::ReadReqRH;
203  IOFile *m_io = nullptr;
204 
205  void Done(int result) override {
206  m_io->ReadEnd(result, this);
207  }
208  };
209 
211 
212  auto *rh = new ZHandler(ObtainReadSid(), &iocb);
213  rh->m_io = this;
214 
215  TRACEIO(Dump, "Read() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " off: " << off << " size: " << size);
216 
217  int retval = ReadBegin(buff, off, size, rh);
218  if (retval != -EWOULDBLOCK)
219  {
220  rh->Done(retval);
221  }
222 }

References XrdPfc::ReadReqRH::ReadReqRH(), Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ ReadV() [1/2]

int IOFile::ReadV ( const XrdOucIOVec readV,
int  n 
)
overridevirtual

Pass ReadV request to the corresponding File object.

Reimplemented from XrdOucCacheIO.

Definition at line 304 of file XrdPfcIOFile.cc.

305 {
307 
308  auto *rh = new ReadReqRHCond(ObtainReadSid(), nullptr);
309 
310  TRACEIO(Dump, "ReadV() sync " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " n_chunks: " << n);
311 
312  rh->m_cond.Lock();
313  int retval = ReadVBegin(readV, n, rh);
314  if (retval == -EWOULDBLOCK)
315  {
316  rh->m_cond.Wait();
317  retval = rh->m_retval;
318  }
319  rh->m_cond.UnLock();
320  return ReadVEnd(retval, rh);
321 }

References Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::ReadReqRH::m_seq_id, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ ReadV() [2/2]

void IOFile::ReadV ( XrdOucCacheIOCB iocb,
const XrdOucIOVec readV,
int  rnum 
)
overridevirtual

Perform an asynchronous vector read (defaults to synchronous).

Parameters
iocbreference to the callback object that receives the result. All results are returned via this object's Done() method. If the caller holds any locks they must be recursive locks as the callback may occur on the calling thread.
readVpointer to a vector of read requests.
rnumthe number of elements in the vector.

Reimplemented from XrdOucCacheIO.

Definition at line 324 of file XrdPfcIOFile.cc.

325 {
326  struct ZHandler : ReadReqRH
327  { using ReadReqRH::ReadReqRH;
328  IOFile *m_io = nullptr;
329 
330  void Done(int result) override { m_io-> ReadVEnd(result, this); }
331  };
332 
334 
335  auto *rh = new ZHandler(ObtainReadSid(), &iocb);
336  rh->m_io = this;
337 
338  TRACEIO(Dump, "ReadV() async " << this << " sid: " << Xrd::hex1 << rh->m_seq_id << " n_chunks: " << n);
339 
340  int retval = ReadVBegin(readV, n, rh);
341  if (retval != -EWOULDBLOCK)
342  {
343  rh->Done(retval);
344  }
345 }

References XrdPfc::ReadReqRH::ReadReqRH(), XrdOucCacheIOCB::Done(), Xrd::hex1, XrdPfc::IO::m_active_read_reqs, XrdPfc::ReadReqRH::m_seq_id, XrdPfc::IO::ObtainReadSid(), and TRACEIO.

+ Here is the call graph for this function:

◆ Update()

void IOFile::Update ( XrdOucCacheIO iocp)
overridevirtual

Update the originally passed XrdOucCacheIO object with the object passed. All future uses underlying XrdOucCacheIO object must now use this object. Update() is called when Prepare() indicated that the file should not be physically opened and a file method was invoked in the XrdOucCacheIO passed to Attach(). When this occurs, the file is actually opened and Update() called to replace the original XrdOucCacheIO object with one that uses the newly opened file.

Parameters
iocpreference to the new XrdOucCacheIO object.

Reimplemented from XrdPfc::IO.

Definition at line 146 of file XrdPfcIOFile.cc.

147 {
148  IO::Update(iocp);
149  m_file->ioUpdated(this);
150 }
void ioUpdated(IO *io)
Notification from IO that it has been updated (remote open).
Definition: XrdPfcFile.cc:179
void Update(XrdOucCacheIO &iocp) override
Definition: XrdPfcIO.cc:16

References XrdPfc::File::ioUpdated(), and XrdPfc::IO::Update().

+ Here is the call graph for this function:

The documentation for this class was generated from the following files: