Performance difference between Poco::FileStream and low level windows functions

[micheleselea]

Expected behavior

Poco::FileStream under windows internally use low level CreateFile WriteFile functions. I think should have almost the same performance in writing files

Actual behavior

What I found is that (with an test USB memory) using Poco::FileStream for writing files of 2MB each I can write about 1 file/sec. Using directly the WriteFile winfows functions I can write about 5 files/sec

Steps to reproduce the problem

code to test

                std::string s;
		s.resize(2048 * 1024);
		int iMaxIter = 50;
		for (int i = 0; i < iMaxIter; i++) {
			printf("%d\n", i);
			Poco::FileStream fout("f:\\test.bmp", std::ios::binary| std::ios::trunc);
			fout.write(s.data(),s.length());
		}

std::string s;
		s.resize(2048 * 1024);
		int iMaxIter = 50;
		for (int i = 0; i < iMaxIter; i++) {
			printf("%d\n", i);
			HANDLE hFile = CreateFile("f:\\test.bmp",                // name of the write
				GENERIC_WRITE,          // open for writing
				0,                      // do not share
				NULL,                   // default security
				CREATE_ALWAYS,             // create new file only
				FILE_ATTRIBUTE_NORMAL,  // normal file
				NULL);
			if (hFile == INVALID_HANDLE_VALUE)
				continue;

			DWORD dwBytesToWrite = (DWORD)s.length();
			DWORD dwBytesWritten = 0;

			BOOL bErrorFlag = WriteFile(
				hFile,           // open file handle
				s.data(),      // start of data to write
				dwBytesToWrite,  // number of bytes to write
				&dwBytesWritten, // number of bytes that were written
				NULL);

			CloseHandle(hFile);
		}

POCO version

1.9.1

Compiler and version

visual studio 2015

Operating system and version

windows 10

Other relevant information

[micheleselea]

I think I found the problem could be here

FileStreamBuf::FileStreamBuf():
	BufferedBidirectionalStreamBuf(BUFFER_SIZE, std::ios::in | std::ios::out),
	_handle(INVALID_HANDLE_VALUE),
	_pos(0)
{
}

BUFFER_SIZE is 4096 so the WriteFile windows function for 2MB data is called several time not in one time only. I check if I can change it deal with it, any suggestions? @obiltschnig @aleks-f

[obiltschnig]

Try to increase it and see if you find a sweet spot between buffer size and performance (e.g., 64 K). Windows machines have plenty of RAM these days. Alternatively, we could make BUFFER_SIZE configurable also.

[micheleselea]

I do not understand one thing: why we use here a BasicBufferedBidirectionalStreamBuf? I don't think we made some internal buffer copy of the data to be written, isn't it? Probably we should manage code to make it call directly the WriteToDevice function with buffer from application

[micheleselea]

using BUFFER_SIZE of 64*1024 I reach the performance of low level functions

[obiltschnig]

With iostreams you cannot pass the client buffer directly to WriteToDevice, unfortunately.

[github-actions]

This issue is stale because it has been open for 365 days with no activity.