Repository navigation
[Breaking change] FileStream does not synchronize file offset with OS anymore #50860
Description
Before .NET 6, FileStream was using expensive syscall called SetFilePointer to synchronize it's private offset with Windows OS.
A blog post from Windows Performance Team from 2008 calls it an anachronism:
The old DOS SetFilePointer API is an anachronism. One should specify the file offset in the overlapped structure even for synchronous I/O. It should never be necessary to resort to the hack of having private file handles for each thread.
In order to improve the performance of async reads and writes and solve the following issues:
- Win32 FileStream will issue a seek on every ReadAsync call #16354 Win32 FileStream will issue a seek on every ReadAsync call
- FileStream.Windows useAsync WriteAsync calls blocking apis #25905 FileStream.Windows useAsync WriteAsync calls blocking apis
FileStream is no longer doing that (#49975) and the offset is just kept in memory. This has allowed for up to two times faster ReadAsync and up to five time faster WriteAsync! See #49975 for full details.
FileStream.Position always returns the current offset, but if the user obtains the file handle via call to FileStream.SafeFileHandle and asks the OS for the current file offset of the given handle by performing a syscall, the value will return 0.
[DllImport("kernel32.dll")]
private static extern bool SetFilePointerEx(SafeFileHandle hFile, long liDistanceToMove, out long lpNewFilePointer, uint dwMoveMethod);
byte[] bytes = new byte[10_000];
string path = Path.Combine(Path.GetTempPath(), Path.GetTempFileName());
using (FileStream fs = new FileStream(path, FileMode.Create, FileAccess.ReadWrite, FileShare.None, bufferSize: 4096, useAsync: true))
{
SafeFileHandle handle = fs.SafeFileHandle;
await fs.WriteAsync(bytes, 0, bytes.Length);
Console.WriteLine(fs.Position);
if (SetFilePointerEx(handle, 0, out long currentOffset, 1 /* get current offset */))
{
Console.WriteLine(currentOffset);
}
}Pre-change behavior:
10000
10000
Post-change behavior:
10000
0
It works in the other direction as well: if the user obtains filehandle and performs a syscall that moves the file offset, the offset returned by FileStream.Position won't be changed.
To enable the .NET 5 behavior, users can specify an AppContext switch or an environment variable:
{
"configProperties": {
"System.IO.UseNet5CompatFileStream": true
}
}
set DOTNET_SYSTEM_IO_USENET5COMPATFILESTREAM=1@dotnet/compat @stephentoub @jozkee @carlossanlop @jeffhandley
FWIW I've ensured that this change is not breaking ASP.NET (dotnet/aspnetcore#31441), WinForms (dotnet/winforms#4756) and the SDK (dotnet/sdk#16684)
Activity
carlossanlop commentedon Apr 20, 2021
@adamsitnik since both #16354 and #25905 got fixed, is there anything else we should address before we close this issue?
Since this is a breaking change that needs to be documented, I opened this issue in dotnet/docs: dotnet/docs#23857
jozkee commentedon May 5, 2021
The current code doesn't actually shows that the Position is not being synchronized with the OS because you are accessing the
SafeFileHandlewhich issues a Seek before returning the handle and thus, the OS knows that you changed the position.We should expose the handle before writing:
using (FileStream fs = new FileStream(path, FileMode.Create, FileAccess.ReadWrite, FileShare.None, bufferSize: 4096, useAsync: true)) { + SafeFileHandle handle = fs.SafeFileHandle; await fs.WriteAsync(bytes, 0, bytes.Length); Console.WriteLine(fs.Position); + if (SetFilePointerEx(handle, 0, out long currentOffset, 1 /* get current offset */)) - if (SetFilePointerEx(fs.SafeFileHandle, 0, out long currentOffset, 1 /* get current offset */)) { Console.WriteLine(currentOffset); Console.WriteLine(fs.Position); } }adamsitnik commentedon May 5, 2021
@jozkee great point! It just shows how hard it is to hit the breaking change scenario ;)
adamsitnik commentedon Aug 11, 2021
Closing (dotnet/docs#24060)