readme.md
1<!--
2 Licensed to the Apache Software Foundation (ASF) under one
3 or more contributor license agreements. See the NOTICE file
4 distributed with this work for additional information
5 regarding copyright ownership. The ASF licenses this file
6 to you under the Apache License, Version 2.0 (the
7 "License"); you may not use this file except in compliance
8 with the License. You may obtain a copy of the License at
9
10 http://www.apache.org/licenses/LICENSE-2.0
11
12 Unless required by applicable law or agreed to in writing,
13 software distributed under the License is distributed on an
14 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15 KIND, either express or implied. See the License for the
16 specific language governing permissions and limitations
17 under the License.
18-->
19
20# Document save path — atomic replace (Issue 126990)
21
22Notes on how `SfxMedium` writes a document over an existing file, and why it is
23written the way it is. The short version: **never truncate the user's document
24before the replacement exists and is on the disk.**
25
26## The bug this addresses
27
28Users reported opening a previously saved document and finding it full of `#`
29characters — hundreds of reports over many years, always after a crash, freeze,
30forced shutdown or power loss. The files were not damaged ODF; they were a run
31of zero bytes of roughly the right length. With no `PK` zip header at offset 0,
32type detection falls through to the plain text filter, and `0x00` renders as `#`.
33
34The zeros were never written by us. Two defects combined to produce them:
35
361. `TransactedTransferForFS_Impl()` truncated the target to zero (via the `Size`
37 property) and then streamed the new contents into it. Despite the name,
38 nothing was transacted — the document was destroyed at the moment of
39 truncation, and everything after that point was a window in which a crash
40 cost the whole file.
412. Nothing forced the data to the medium. `osl_closeFile()` flushes osl's own
42 buffer and calls `CloseHandle()`; it never calls `FlushFileBuffers()`/`fsync()`.
43 So the bytes sat in the OS page cache while the save reported success.
44
45File systems that journal metadata but not data — NTFS, HFS+, XFS, and ext4 with
46delayed allocation — commit the new file *size* while the data is still only in
47the cache. Crash in that window and the file reads back as N zeros. Because the
48save had already reported success, users acted on it and lost the document.
49
50## What the code does now
51
52Two changes, and they are **not independent — the flush is a prerequisite for the
53rename.** Renaming a replacement whose data is still in the page cache would swap
54in a file that the same crash turns into zeros, just under a different name.
55
56- `ucb/source/ucp/file/shell.cxx`, `shell::write()` — syncs before reporting
57 success, and reports a sync failure as a write failure rather than a silent
58 success.
59- `docfile.cxx`, `lcl_ReplaceTargetAtomically()` — streams the new contents into a
60 sibling of the target and renames it over the target. The previous document
61 stays intact until one atomic operation swaps in the new one, so an interruption
62 now costs at most the most recent save instead of the whole document.
63
64Note that configmgr has always written `registrymodifications.xcu` this way. The
65document save path was the odd one out.
66
67## Constraints — do not "simplify" these away
68
69- **The replacement must be a sibling of the target.** `osl_moveFile()` passes
70 `MOVEFILE_COPY_ALLOWED`, which silently degrades a cross-volume move into a
71 non-atomic copy+delete, and POSIX `rename()` cannot cross a mount point at all.
72 `SfxMedium::CreateTempFile()` produces temporaries in the *system* temp
73 directory — reusing one of those would quietly destroy the atomicity.
74- **The in-place path is kept as a fallback, not dead code.** A rename needs
75 delete access on the target; an in-place rewrite does not. A process holding
76 the document open — antivirus, a search indexer — would otherwise turn a save
77 that used to work into a hard failure. That is a worse regression than the rare
78 corruption this fixes.
79- **Symlinked targets deliberately take the fallback.** An in-place rewrite
80 follows the link and updates what it points at; a rename would replace the link
81 itself with a regular file.
82- **`bTransactStarted` and `UseBackupToRestore_Impl()` stay live** — they are
83 reachable through the fallback.
84
85## Known limitations
86
87- **ACLs, alternate data streams, creation time.** The renamed file carries the
88 sibling's security descriptor, not the original's. Where ACLs are inherited
89 from the containing folder — the common case — the result is identical and
90 nothing is lost; it only differs for explicit per-file ACEs and ADS. Win32
91 `ReplaceFile()` is the primitive that preserves all of it, but it is not exposed
92 by osl, and adding it would mean a new exported sal API.
93- **POSIX rename durability.** `rename()` is atomic, but the rename itself is not
94 durable without an `fsync` of the containing directory, and osl has no
95 directory-sync API. On Windows `MOVEFILE_WRITE_THROUGH` covers this. The POSIX
96 failure mode without it is "last save lost", not "document destroyed".
97- **macOS `fsync` is weaker than it looks** — it does not flush the drive write
98 cache; `fcntl(F_FULLFSYNC)` would be required.
99- **Peak disk space on the target volume roughly doubles.** The original and the
100 replacement now coexist until the rename, where the old code held only the file
101 being rewritten. A large save onto a nearly full volume can therefore hit
102 `ENOSPC` where it previously did not; that falls back to the in-place path.
103 Conversely the disk-full case is now *safer*: the original is no longer
104 destroyed before it is known that the new data fits.
105- **The backup is still not durable.** `DoInternalBackup_Impl()` copies via
106 `transferContent`, which routes to the file UCP's `copy` rather than
107 `shell::write`, so it does not get the sync.
108
109## Verifying a build
110
111The interesting calls are exported Win32 APIs, so this works on a release build
112with no PDBs and no debug-level rebuild. Under `cdb`:
113
114```
115bu KERNELBASE!MoveFileExW ".echo >>> MoveFileExW; du @rcx; du @rdx; gu; r rax; gc"
116bu KERNELBASE!FlushFileBuffers ".echo >>> FlushFileBuffers; gc"
117```
118
119A healthy overwrite save shows flush(es), then a rename whose source and
120destination are **in the same directory**, with `r8=b`
121(`REPLACE_EXISTING|COPY_ALLOWED|WRITE_THROUGH`) identifying it as `osl_moveFile`:
122
123```
124>>> FlushFileBuffers
125>>> MoveFileExW
126 C:\Users\...\Documents\mydoc0.odt
127 C:\Users\...\Documents\mydoc.odt
128 rax=1
129```
130
131Things that will waste your time otherwise:
132
133- **Save As to a new file never takes the atomic path**, by design — the helper is
134 behind `bOverWrite && IsDocument(aDest)` and there is nothing yet to protect.
135 Only an overwrite exercises it.
136- **`MoveFileExW` traffic is normal background noise** — configmgr writes its store
137 the same way, with the same flags. Always read the paths before concluding
138 anything; `\user\` is config, not the document.
139- **The breakpoints fire on entry**, so a *failed* rename looks identical to a
140 successful one. Capture `rax` (as above) or the fallback is invisible. Without
141 it, the fallback's tell is one extra `FlushFileBuffers` immediately after the
142 document rename.
143- **`SetEndOfFile` is not a fallback signal** — it also fires when `shell::write`
144 truncates the sibling that `TempFile` has already created.
145
146To exercise the fallback deliberately, hold the target open against delete from
147another process while saving; the rename fails and the save must still succeed:
148
149```powershell
150$f = [System.IO.File]::Open($p, 'Open', 'Read', 'ReadWrite') # ReadWrite withholds Delete
151```
152