# HG changeset patch # User Artem Tikhomirov # Date 1368641409 -7200 # Node ID 65c01508f0026febb7330947ba415f3fc7c3a72f # Parent 5e0313485eef4e449fc8d7b184bf2a6f387795b5 Rollback support for commands that modify repository. Strategy to keep complete copy of a file being changed diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/core/HgAddRemoveCommand.java --- a/src/org/tmatesoft/hg/core/HgAddRemoveCommand.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/core/HgAddRemoveCommand.java Wed May 15 20:10:09 2013 +0200 @@ -18,11 +18,14 @@ import java.util.LinkedHashSet; +import org.tmatesoft.hg.internal.COWTransaction; import org.tmatesoft.hg.internal.DirstateBuilder; import org.tmatesoft.hg.internal.DirstateReader; import org.tmatesoft.hg.internal.Internals; +import org.tmatesoft.hg.internal.Transaction; import org.tmatesoft.hg.repo.HgManifest.Flags; import org.tmatesoft.hg.repo.HgRepository; +import org.tmatesoft.hg.repo.HgRepositoryLock; import org.tmatesoft.hg.repo.HgRuntimeException; import org.tmatesoft.hg.util.CancelSupport; import org.tmatesoft.hg.util.CancelledException; @@ -94,9 +97,12 @@ * Perform scheduled addition/removal * * @throws HgException subclass thereof to indicate specific issue with the command arguments or repository state + * @throws HgRepositoryLockException if failed to lock the repo for modifications * @throws CancelledException if execution of the command was cancelled */ - public void execute() throws HgException, CancelledException { + public void execute() throws HgException, HgRepositoryLockException, CancelledException { + final HgRepositoryLock wdLock = repo.getWorkingDirLock(); + wdLock.acquire(); try { final ProgressSupport progress = getProgressSupport(null); final CancelSupport cancellation = getCancelSupport(null, true); @@ -117,11 +123,24 @@ progress.worked(1); cancellation.checkCancelled(); } - dirstateBuilder.serialize(); + Transaction.Factory trFactory = new COWTransaction.Factory(); + Transaction tr = trFactory.create(repo); + try { + dirstateBuilder.serialize(tr); + tr.commit(); + } catch (RuntimeException ex) { + tr.rollback(); + throw ex; + } catch (HgException ex) { + tr.rollback(); + throw ex; + } progress.worked(1); progress.done(); } catch (HgRuntimeException ex) { throw new HgLibraryFailureException(ex); + } finally { + wdLock.release(); } } } diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/core/HgCommitCommand.java --- a/src/org/tmatesoft/hg/core/HgCommitCommand.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/core/HgCommitCommand.java Wed May 15 20:10:09 2013 +0200 @@ -23,10 +23,13 @@ import java.util.ArrayList; import org.tmatesoft.hg.internal.ByteArrayChannel; +import org.tmatesoft.hg.internal.COWTransaction; import org.tmatesoft.hg.internal.CommitFacility; +import org.tmatesoft.hg.internal.CompleteRepoLock; import org.tmatesoft.hg.internal.Experimental; import org.tmatesoft.hg.internal.FileContentSupplier; import org.tmatesoft.hg.internal.Internals; +import org.tmatesoft.hg.internal.Transaction; import org.tmatesoft.hg.repo.HgChangelog; import org.tmatesoft.hg.repo.HgDataFile; import org.tmatesoft.hg.repo.HgInternals; @@ -86,6 +89,7 @@ /** * @throws HgException subclass thereof to indicate specific issue with the command arguments or repository state + * @throws HgRepositoryLockException if failed to lock the repo for modifications * @throws IOException propagated IO errors from status walker over working directory * @throws CancelledException if execution of the command was cancelled */ @@ -93,6 +97,8 @@ if (message == null) { throw new HgBadArgumentException("Shall supply commit message", null); } + final CompleteRepoLock repoLock = new CompleteRepoLock(repo); + repoLock.acquire(); try { int[] parentRevs = new int[2]; detectParentFromDirstate(parentRevs); @@ -127,7 +133,18 @@ } cf.branch(detectBranch()); cf.user(detectUser()); - newRevision = cf.commit(message); + Transaction.Factory trFactory = new COWTransaction.Factory(); + Transaction tr = trFactory.create(repo); + try { + newRevision = cf.commit(message, tr); + tr.commit(); + } catch (RuntimeException ex) { + tr.rollback(); + throw ex; + } catch (HgException ex) { + tr.rollback(); + throw ex; + } // TODO toClear list is awful for (FileContentSupplier fcs : toClear) { fcs.done(); @@ -135,6 +152,8 @@ return new Outcome(Kind.Success, "Commit ok"); } catch (HgRuntimeException ex) { throw new HgLibraryFailureException(ex); + } finally { + repoLock.release(); } } diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/core/HgRevertCommand.java --- a/src/org/tmatesoft/hg/core/HgRevertCommand.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/core/HgRevertCommand.java Wed May 15 20:10:09 2013 +0200 @@ -21,13 +21,16 @@ import java.util.LinkedHashSet; import java.util.Set; +import org.tmatesoft.hg.internal.COWTransaction; import org.tmatesoft.hg.internal.CsetParamKeeper; import org.tmatesoft.hg.internal.DirstateBuilder; import org.tmatesoft.hg.internal.DirstateReader; import org.tmatesoft.hg.internal.Internals; +import org.tmatesoft.hg.internal.Transaction; import org.tmatesoft.hg.repo.HgManifest; import org.tmatesoft.hg.repo.HgManifest.Flags; import org.tmatesoft.hg.repo.HgRepository; +import org.tmatesoft.hg.repo.HgRepositoryLock; import org.tmatesoft.hg.repo.HgRuntimeException; import org.tmatesoft.hg.util.CancelSupport; import org.tmatesoft.hg.util.CancelledException; @@ -99,6 +102,8 @@ * @throws CancelledException if execution of the command was cancelled */ public void execute() throws HgException, CancelledException { + final HgRepositoryLock wdLock = repo.getWorkingDirLock(); + wdLock.acquire(); try { final ProgressSupport progress = getProgressSupport(null); final CancelSupport cancellation = getCancelSupport(null, true); @@ -155,11 +160,25 @@ progress.worked(1); cancellation.checkCancelled(); } - dirstateBuilder.serialize(); + Transaction.Factory trFactory = new COWTransaction.Factory(); + Transaction tr = trFactory.create(repo); + try { + // TODO same code in HgAddRemoveCommand and similar in HgCommitCommand + dirstateBuilder.serialize(tr); + tr.commit(); + } catch (RuntimeException ex) { + tr.rollback(); + throw ex; + } catch (HgException ex) { + tr.rollback(); + throw ex; + } progress.worked(1); progress.done(); } catch (HgRuntimeException ex) { throw new HgLibraryFailureException(ex); + } finally { + wdLock.release(); } } } diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/COWTransaction.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/org/tmatesoft/hg/internal/COWTransaction.java Wed May 15 20:10:09 2013 +0200 @@ -0,0 +1,176 @@ +/* + * Copyright (c) 2013 TMate Software Ltd + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * For information on how to redistribute this software under + * the terms of a license other than GNU General Public License + * contact TMate Software at support@hg4j.com + */ +package org.tmatesoft.hg.internal; + +import java.io.File; +import java.io.IOException; +import java.util.Iterator; +import java.util.LinkedList; +import java.util.List; + +import org.tmatesoft.hg.core.HgIOException; +import org.tmatesoft.hg.core.SessionContext; + +/** + * This transaction strategy makes a copy of original file and breaks origin hard links, if any. + * Changes are directed to actual repository files. + * + * On commit, remove all backup copies + * On rollback, move all backup files in place of original + * + * @author Artem Tikhomirov + * @author TMate Software Ltd. + */ +public final class COWTransaction extends Transaction { + + private final FileUtils fileHelper; + private final List entries = new LinkedList(); + + public COWTransaction(SessionContext.Source ctxSource) { + fileHelper = new FileUtils(ctxSource.getSessionContext().getLog()); + } + + @Override + public File prepare(File f) throws HgIOException { + if (!f.exists()) { + record(f, null); + return f; + } + if (known(f)) { + return f; + } + final File parentDir = f.getParentFile(); + assert parentDir.canWrite(); + File copy = new File(parentDir, f.getName() + ".hg4j.copy"); + fileHelper.copy(f, copy); + final long lm = f.lastModified(); + copy.setLastModified(lm); + File backup = new File(parentDir, f.getName() + ".hg4j.orig"); + if (backup.exists()) { + backup.delete(); + } + if (!f.renameTo(backup)) { + throw new HgIOException(String.format("Failed to backup %s to %s", f.getName(), backup.getName()), backup); + } + if (!copy.renameTo(f)) { + throw new HgIOException(String.format("Failed to bring on-write copy in place (%s to %s)", copy.getName(), f.getName()), copy); + } + f.setLastModified(lm); + record(f, backup); + return f; + } + + @Override + public File prepare(File origin, File backup) throws HgIOException { + if (known(origin)) { + return origin; + } + fileHelper.copy(origin, backup); + final RollbackEntry e = record(origin, backup); + e.keepBackup = true; + return origin; + } + + @Override + public void done(File f) throws HgIOException { + find(f).success = true; + } + + @Override + public void failure(File f, IOException ex) { + find(f).failure = ex; + } + + // XXX custom exception for commit and rollback to hold information about files rolled back + + @Override + public void commit() throws HgIOException { + for (Iterator it = entries.iterator(); it.hasNext();) { + RollbackEntry e = it.next(); + assert e.success; + if (e.failure != null) { + throw new HgIOException("Can't close transaction with a failure.", e.failure, e.origin); + } + if (!e.keepBackup && e.backup != null) { + e.backup.delete(); + } + it.remove(); + } + } + + @Override + public void rollback() throws HgIOException { + LinkedList success = new LinkedList(); + for (Iterator it = entries.iterator(); it.hasNext();) { + RollbackEntry e = it.next(); + e.origin.delete(); + if (e.backup != null) { + if (!e.backup.renameTo(e.origin)) { + String msg = String.format("Transaction rollback failed, could not rename backup %s back to %s", e.backup.getName(), e.origin.getName()); + throw new HgIOException(msg, e.origin); + } + } + success.add(e); + it.remove(); + } + } + + private RollbackEntry record(File origin, File backup) { + final RollbackEntry e = new RollbackEntry(origin, backup); + entries.add(e); + return e; + } + + private boolean known(File f) { + for (RollbackEntry e : entries) { + if (e.origin.equals(f)) { + return true; + } + } + return false; + } + private RollbackEntry find(File f) { + for (RollbackEntry e : entries) { + if (e.origin.equals(f)) { + return e; + } + } + assert false; + return new RollbackEntry(f,f); + } + + private static class RollbackEntry { + public final File origin; + public final File backup; // may be null to indicate file didn't exist + public boolean success = false; + public IOException failure = null; + public boolean keepBackup = false; + + public RollbackEntry(File o, File b) { + origin = o; + backup = b; + } + } + + public static class Factory implements Transaction.Factory { + + public Transaction create(SessionContext.Source ctxSource) { + return new COWTransaction(ctxSource); + } + + } +} diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/CommitFacility.java --- a/src/org/tmatesoft/hg/internal/CommitFacility.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/internal/CommitFacility.java Wed May 15 20:10:09 2013 +0200 @@ -19,6 +19,7 @@ import static org.tmatesoft.hg.repo.HgRepository.DEFAULT_BRANCH_NAME; import static org.tmatesoft.hg.repo.HgRepository.NO_REVISION; import static org.tmatesoft.hg.repo.HgRepositoryFiles.Branch; +import static org.tmatesoft.hg.repo.HgRepositoryFiles.UndoBranch; import static org.tmatesoft.hg.util.LogFacility.Severity.Error; import java.io.File; @@ -94,7 +95,9 @@ user = userName; } - public Nodeid commit(String message) throws HgIOException, HgRepositoryLockException { + // this method doesn't roll transaction back in case of failure, caller's responsibility + // this method expects repository to be locked, if needed + public Nodeid commit(String message, Transaction transaction) throws HgIOException, HgRepositoryLockException { final HgChangelog clog = repo.getRepo().getChangelog(); final int clogRevisionIndex = clog.getRevisionCount(); ManifestRevision c1Manifest = new ManifestRevision(null, null); @@ -161,7 +164,7 @@ // that would attempt to access newly added file after commit would fail // (despite the fact the file is in there) } - RevlogStreamWriter fileWriter = new RevlogStreamWriter(repo, contentStream); + RevlogStreamWriter fileWriter = new RevlogStreamWriter(repo, contentStream, transaction); Nodeid fileRev = fileWriter.addRevision(bac.toArray(), clogRevisionIndex, fp.first(), fp.second()); newManifestRevision.put(df.getPath(), fileRev); touchInDirstate.add(df.getPath()); @@ -172,7 +175,7 @@ for (Map.Entry me : newManifestRevision.entrySet()) { manifestBuilder.add(me.getKey().toString(), me.getValue()); } - RevlogStreamWriter manifestWriter = new RevlogStreamWriter(repo, repo.getImplAccess().getManifestStream()); + RevlogStreamWriter manifestWriter = new RevlogStreamWriter(repo, repo.getImplAccess().getManifestStream(), transaction); Nodeid manifestRev = manifestWriter.addRevision(manifestBuilder.build(), clogRevisionIndex, manifestParents.first(), manifestParents.second()); // // Changelog @@ -181,7 +184,7 @@ changelogBuilder.branch(branch == null ? DEFAULT_BRANCH_NAME : branch); changelogBuilder.user(String.valueOf(user)); byte[] clogContent = changelogBuilder.build(manifestRev, message); - RevlogStreamWriter changelogWriter = new RevlogStreamWriter(repo, repo.getImplAccess().getChangelogStream()); + RevlogStreamWriter changelogWriter = new RevlogStreamWriter(repo, repo.getImplAccess().getChangelogStream(), transaction); Nodeid changesetRev = changelogWriter.addRevision(clogContent, clogRevisionIndex, p1Commit, p2Commit); // TODO move fncache update to an external facility, along with dirstate and bookmark update if (!newlyAddedFiles.isEmpty() && repo.fncacheInUse()) { @@ -201,14 +204,19 @@ } String oldBranchValue = DirstateReader.readBranch(repo); String newBranchValue = branch == null ? DEFAULT_BRANCH_NAME : branch; + // TODO undo.dirstate and undo.branch as described in http://mercurial.selenic.com/wiki/FileFormats#undo..2A if (!oldBranchValue.equals(newBranchValue)) { - File branchFile = repo.getRepositoryFile(Branch); + File branchFile = transaction.prepare(repo.getRepositoryFile(Branch), repo.getRepositoryFile(UndoBranch)); FileOutputStream fos = null; try { fos = new FileOutputStream(branchFile); fos.write(newBranchValue.getBytes(EncodingHelper.getUTF8())); fos.flush(); + fos.close(); + fos = null; + transaction.done(branchFile); } catch (IOException ex) { + transaction.failure(branchFile, ex); repo.getLog().dump(getClass(), Error, ex, "Failed to write branch information, error ignored"); } finally { try { @@ -220,7 +228,7 @@ } } } - // bring dirstate up to commit state + // bring dirstate up to commit state, TODO share this code with HgAddRemoveCommand final DirstateBuilder dirstateBuilder = new DirstateBuilder(repo); dirstateBuilder.fillFrom(new DirstateReader(repo, new Path.SimpleSource())); for (Path p : removals) { @@ -230,12 +238,11 @@ dirstateBuilder.recordUncertain(p); } dirstateBuilder.parents(changesetRev, Nodeid.NULL); - dirstateBuilder.serialize(); + dirstateBuilder.serialize(transaction); // update bookmarks if (p1Commit != NO_REVISION || p2Commit != NO_REVISION) { repo.getRepo().getBookmarks().updateActive(p1Cset, p2Cset, changesetRev); } - // TODO undo.dirstate and undo.branch as described in http://mercurial.selenic.com/wiki/FileFormats#undo..2A // TODO Revisit: might be reasonable to send out a "Repo changed" notification, to clear // e.g. cached branch, tags and so on, not to rely on file change detection methods? // The same notification might come useful once Pull is implemented diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/CompleteRepoLock.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/org/tmatesoft/hg/internal/CompleteRepoLock.java Wed May 15 20:10:09 2013 +0200 @@ -0,0 +1,73 @@ +/* + * Copyright (c) 2013 TMate Software Ltd + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * For information on how to redistribute this software under + * the terms of a license other than GNU General Public License + * contact TMate Software at support@hg4j.com + */ +package org.tmatesoft.hg.internal; + +import static org.tmatesoft.hg.util.LogFacility.Severity.Error; + +import org.tmatesoft.hg.core.HgRepositoryLockException; +import org.tmatesoft.hg.repo.HgRepository; +import org.tmatesoft.hg.repo.HgRepositoryLock; +import org.tmatesoft.hg.util.LogFacility; + +/** + * Helper to lock both storage and working directory + * + * @author Artem Tikhomirov + * @author TMate Software Ltd. + */ +public final class CompleteRepoLock { + + private final HgRepository repo; + private HgRepositoryLock wdLock, storeLock; + + public CompleteRepoLock(HgRepository hgRepo) { + repo = hgRepo; + } + + public void acquire() throws HgRepositoryLockException { + wdLock = repo.getWorkingDirLock(); + storeLock = repo.getStoreLock(); + wdLock.acquire(); + try { + storeLock.acquire(); + } catch (HgRepositoryLockException ex) { + try { + wdLock.release(); + } catch (HgRepositoryLockException e2) { + final LogFacility log = repo.getSessionContext().getLog(); + log.dump(getClass(), Error, e2, "Nested exception ignored once failed to acquire store lock"); + } + throw ex; + } + + } + + public void release() throws HgRepositoryLockException { + try { + storeLock.release(); + } catch (HgRepositoryLockException ex) { + try { + wdLock.release(); + } catch (HgRepositoryLockException e2) { + final LogFacility log = repo.getSessionContext().getLog(); + log.dump(getClass(), Error, e2, "Nested exception ignored when releasing working directory lock"); + } + throw ex; + } + wdLock.release(); + } +} diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/DataAccessProvider.java --- a/src/org/tmatesoft/hg/internal/DataAccessProvider.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/internal/DataAccessProvider.java Wed May 15 20:10:09 2013 +0200 @@ -29,6 +29,7 @@ import java.nio.MappedByteBuffer; import java.nio.channels.FileChannel; +import org.tmatesoft.hg.core.HgIOException; import org.tmatesoft.hg.core.SessionContext; import org.tmatesoft.hg.util.LogFacility; @@ -80,7 +81,7 @@ return new DataAccess(); } try { - FileChannel fc = new FileInputStream(f).getChannel(); + FileChannel fc = new FileInputStream(f).getChannel(); // FIXME SHALL CLOSE FIS, not only channel long flen = fc.size(); if (!shortRead && flen > mapioMagicBoundary) { // TESTS: bufLen of 1024 was used to test MemMapFileAccess @@ -100,12 +101,26 @@ return new DataAccess(); // non-null, empty. } - public DataSerializer createWriter(File f, boolean createNewIfDoesntExist) { + public DataSerializer createWriter(final Transaction tr, File f, boolean createNewIfDoesntExist) throws HgIOException { if (!f.exists() && !createNewIfDoesntExist) { return new DataSerializer(); } try { - return new StreamDataSerializer(context.getLog(), new FileOutputStream(f, true)); + final File transactionFile = tr.prepare(f); + return new StreamDataSerializer(context.getLog(), new FileOutputStream(transactionFile, true)) { + @Override + public void done() { + super.done(); + // FIXME invert RevlogStreamWriter to send DataSource here instead of grabbing DataSerializer + // besides, DataSerializer#done is invoked regardless of whether write was successful or not, + // while Transaction#done() assumes there's no error + try { + tr.done(transactionFile); + } catch (HgIOException ex) { + context.getLog().dump(DataAccessProvider.class, Error, ex, null); + } + } + }; } catch (final FileNotFoundException ex) { context.getLog().dump(getClass(), Error, ex, null); return new DataSerializer() { diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/DirstateBuilder.java --- a/src/org/tmatesoft/hg/internal/DirstateBuilder.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/internal/DirstateBuilder.java Wed May 15 20:10:09 2013 +0200 @@ -16,6 +16,9 @@ */ package org.tmatesoft.hg.internal; +import static org.tmatesoft.hg.repo.HgRepositoryFiles.Dirstate; +import static org.tmatesoft.hg.repo.HgRepositoryFiles.UndoDirstate; + import java.io.File; import java.io.FileOutputStream; import java.io.IOException; @@ -32,7 +35,6 @@ import org.tmatesoft.hg.repo.HgDirstate.Record; import org.tmatesoft.hg.repo.HgInvalidStateException; import org.tmatesoft.hg.repo.HgManifest.Flags; -import org.tmatesoft.hg.repo.HgRepositoryFiles; import org.tmatesoft.hg.util.Path; /** @@ -149,13 +151,15 @@ } } - public void serialize() throws HgIOException { - File dirstateFile = hgRepo.getRepositoryFile(HgRepositoryFiles.Dirstate); + public void serialize(Transaction tr) throws HgIOException { + File dirstateFile = tr.prepare(hgRepo.getRepositoryFile(Dirstate), hgRepo.getRepositoryFile(UndoDirstate)); try { FileChannel dirstate = new FileOutputStream(dirstateFile).getChannel(); serialize(dirstate); dirstate.close(); + tr.done(dirstateFile); } catch (IOException ex) { + tr.failure(dirstateFile, ex); throw new HgIOException("Can't write down new directory state", ex, dirstateFile); } } diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/FileUtils.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/org/tmatesoft/hg/internal/FileUtils.java Wed May 15 20:10:09 2013 +0200 @@ -0,0 +1,98 @@ +/* + * Copyright (c) 2013 TMate Software Ltd + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * For information on how to redistribute this software under + * the terms of a license other than GNU General Public License + * contact TMate Software at support@hg4j.com + */ +package org.tmatesoft.hg.internal; + +import static org.tmatesoft.hg.util.LogFacility.Severity.Debug; + +import java.io.Closeable; +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.io.IOException; +import java.nio.channels.FileChannel; + +import org.tmatesoft.hg.core.HgIOException; +import org.tmatesoft.hg.util.LogFacility; +import org.tmatesoft.hg.util.LogFacility.Severity; + +/** + * + * @author Artem Tikhomirov + * @author TMate Software Ltd. + */ +final class FileUtils { + + private final LogFacility log; + + public static void copyFile(File from, File to) throws HgIOException { + new FileUtils(new StreamLogFacility(Debug, true, System.err)).copy(from, to); + } + + public FileUtils(LogFacility logFacility) { + log = logFacility; + } + + public void copy(File from, File to) throws HgIOException { + FileInputStream fis = null; + FileOutputStream fos = null; + try { + fis = new FileInputStream(from); + fos = new FileOutputStream(to); + FileChannel input = fis.getChannel(); + FileChannel output = fos.getChannel(); + long count = input.size(); + long pos = 0; + int zeroCopied = 0; // flag to prevent hang-up + do { + long c = input.transferTo(pos, count, output); + pos += c; + count -= c; + if (c == 0) { + if (++zeroCopied == 3) { + String m = String.format("Can't copy %s to %s, transferTo copies 0 bytes. Position: %d, bytes left:%d", from.getName(), to.getName(), pos, count); + throw new IOException(m); + } + } else { + // reset + zeroCopied = 0; + } + } while (count > 0); + fos.close(); + fos = null; + fis.close(); + fis = null; + } catch (IOException ex) { + // not in finally because I don't want to loose exception from fos.close() + closeQuietly(fis); + closeQuietly(fos); + String m = String.format("Failed to copy %s to %s", from.getName(), to.getName()); + throw new HgIOException(m, ex, from); + } + } + + public void closeQuietly(Closeable stream) { + if (stream != null) { + try { + stream.close(); + } catch (IOException ex) { + // ignore + log.dump(getClass(), Severity.Warn, ex, "Exception while closing stream quietly"); + } + } + } + +} diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/RevlogStream.java --- a/src/org/tmatesoft/hg/internal/RevlogStream.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/internal/RevlogStream.java Wed May 15 20:10:09 2013 +0200 @@ -29,6 +29,7 @@ import java.util.List; import java.util.zip.Inflater; +import org.tmatesoft.hg.core.HgIOException; import org.tmatesoft.hg.core.Nodeid; import org.tmatesoft.hg.repo.HgInternals; import org.tmatesoft.hg.repo.HgInvalidControlFileException; @@ -101,14 +102,14 @@ return dataAccess.createReader(getDataFile(), false); } - /*package*/ DataSerializer getIndexStreamWriter() { + /*package*/ DataSerializer getIndexStreamWriter(Transaction tr) throws HgIOException { DataAccessProvider dataAccess = repo.getDataAccess(); - return dataAccess.createWriter(indexFile, true); + return dataAccess.createWriter(tr, indexFile, true); } - /*package*/ DataSerializer getDataStreamWriter() { + /*package*/ DataSerializer getDataStreamWriter(Transaction tr) throws HgIOException { DataAccessProvider dataAccess = repo.getDataAccess(); - return dataAccess.createWriter(getDataFile(), true); + return dataAccess.createWriter(tr, getDataFile(), true); } /** diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/RevlogStreamWriter.java --- a/src/org/tmatesoft/hg/internal/RevlogStreamWriter.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/internal/RevlogStreamWriter.java Wed May 15 20:10:09 2013 +0200 @@ -19,9 +19,11 @@ import static org.tmatesoft.hg.internal.Internals.REVLOGV1_RECORD_SIZE; import static org.tmatesoft.hg.repo.HgRepository.NO_REVISION; +import java.io.File; import java.io.IOException; import java.nio.ByteBuffer; +import org.tmatesoft.hg.core.HgIOException; import org.tmatesoft.hg.core.Nodeid; import org.tmatesoft.hg.core.SessionContext; import org.tmatesoft.hg.repo.HgInvalidControlFileException; @@ -38,24 +40,27 @@ private final DigestHelper dh = new DigestHelper(); private final RevlogCompressor revlogDataZip; + private final Transaction transaction; private int lastEntryBase, lastEntryIndex; private byte[] lastEntryContent; private Nodeid lastEntryRevision; private IntMap revisionCache = new IntMap(32); private RevlogStream revlogStream; - public RevlogStreamWriter(SessionContext.Source ctxSource, RevlogStream stream) { + public RevlogStreamWriter(SessionContext.Source ctxSource, RevlogStream stream, Transaction tr) { assert ctxSource != null; assert stream != null; + assert tr != null; revlogDataZip = new RevlogCompressor(ctxSource.getSessionContext()); revlogStream = stream; + transaction = tr; } /** * @return nodeid of added revision */ - public Nodeid addRevision(byte[] content, int linkRevision, int p1, int p2) { + public Nodeid addRevision(byte[] content, int linkRevision, int p1, int p2) throws HgIOException { lastEntryRevision = Nodeid.NULL; int revCount = revlogStream.revisionCount(); lastEntryIndex = revCount == 0 ? NO_REVISION : revCount - 1; @@ -85,7 +90,7 @@ indexFile = dataFile = activeFile = null; try { // - activeFile = indexFile = revlogStream.getIndexStreamWriter(); + activeFile = indexFile = revlogStream.getIndexStreamWriter(transaction); final boolean isInlineData = revlogStream.isInlineData(); HeaderWriter revlogHeader = new HeaderWriter(isInlineData); revlogHeader.length(content.length, compressedLen); @@ -101,7 +106,7 @@ if (isInlineData) { dataFile = indexFile; } else { - dataFile = revlogStream.getDataStreamWriter(); + dataFile = revlogStream.getDataStreamWriter(transaction); } activeFile = dataFile; if (useCompressedData) { @@ -124,12 +129,8 @@ revlogStream.revisionAdded(lastEntryIndex, lastEntryRevision, lastEntryBase, lastEntryOffset); } catch (IOException ex) { String m = String.format("Failed to write revision %d", lastEntryIndex+1, null); - HgInvalidControlFileException t = new HgInvalidControlFileException(m, ex, null); - if (activeFile == dataFile) { - throw revlogStream.initWithDataFile(t); - } else { - throw revlogStream.initWithIndexFile(t); - } + // FIXME proper file in the exception based on activeFile == dataFile || indexFile + throw new HgIOException(m, ex, new File(revlogStream.getDataFileName())); } finally { indexFile.done(); if (dataFile != null && dataFile != indexFile) { diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/internal/Transaction.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/org/tmatesoft/hg/internal/Transaction.java Wed May 15 20:10:09 2013 +0200 @@ -0,0 +1,106 @@ +/* + * Copyright (c) 2013 TMate Software Ltd + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; version 2 of the License. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * For information on how to redistribute this software under + * the terms of a license other than GNU General Public License + * contact TMate Software at support@hg4j.com + */ +package org.tmatesoft.hg.internal; + +import java.io.File; +import java.io.IOException; + +import org.tmatesoft.hg.core.HgIOException; +import org.tmatesoft.hg.core.SessionContext; +import org.tmatesoft.hg.repo.HgInvalidStateException; + +/** + * Implementation strategies possible:
    + *
  • Get a copy, write changes to origin, keep copy as backup till #commit + *

    (-) doesn't break hard links + *

  • Get a copy, write changes to a copy, on commit rename copy to origin. + *

    (-) What if we read newly written data (won't find it); + *

    (-) complex #commit + *

    (+) simple rollback + *

  • Get a copy, rename origin to backup (breaks hard links), rename copy to origin, write changes + *

    (+) Modified file is in place right away; + *

    (+) easy #commit + *

  • Do not copy, just record file size, truncate to that size on rollback + *
  • ...? + *
+ * @author Artem Tikhomirov + * @author TMate Software Ltd. + */ +public abstract class Transaction { + /** + * Record the file is going to be modified during this transaction, obtain actual + * destination to write to. + */ + public abstract File prepare(File f) throws HgIOException; + /** + * overwrites backup if exists, backup is kept after successful {@link #commit()} + */ + public abstract File prepare(File origin, File backup) throws HgIOException; + /** + * Tell that file was successfully processed + */ + public abstract void done(File f) throws HgIOException; + /** + * optional? + */ + public abstract void failure(File f, IOException ex); + /** + * Complete the transaction + */ + public abstract void commit() throws HgIOException; + /** + * Undo all the changes + */ + public abstract void rollback() throws HgIOException; + + public interface Factory { + public Transaction create(SessionContext.Source ctxSource); + } + + public static class NoRollback extends Transaction { + + @Override + public File prepare(File f) throws HgIOException { + return f; + } + + @Override + public File prepare(File origin, File backup) throws HgIOException { + return origin; + } + + @Override + public void done(File f) throws HgIOException { + // no-op + } + + @Override + public void failure(File f, IOException ex) { + // no-op + } + + @Override + public void commit() throws HgIOException { + // no-op + } + + @Override + public void rollback() throws HgIOException { + throw new HgInvalidStateException("This transaction doesn't support rollback"); + } + } +} diff -r 5e0313485eef -r 65c01508f002 src/org/tmatesoft/hg/repo/HgRepositoryFiles.java --- a/src/org/tmatesoft/hg/repo/HgRepositoryFiles.java Tue May 14 17:31:35 2013 +0200 +++ b/src/org/tmatesoft/hg/repo/HgRepositoryFiles.java Wed May 15 20:10:09 2013 +0200 @@ -30,7 +30,8 @@ HgSub(".hgsub"), HgSubstate(".hgsubstate"), LastMessage(false, "last-message.txt"), Bookmarks(false, "bookmarks"), BookmarksCurrent(false, "bookmarks.current"), - Branch(false, "branch"); + Branch(false, "branch"), + UndoBranch(false, "undo.branch"), UndoDirstate(false, "undo.dirstate"); private final String fname; private final boolean livesInWC; diff -r 5e0313485eef -r 65c01508f002 test/org/tmatesoft/hg/test/TestCommit.java --- a/test/org/tmatesoft/hg/test/TestCommit.java Tue May 14 17:31:35 2013 +0200 +++ b/test/org/tmatesoft/hg/test/TestCommit.java Wed May 15 20:10:09 2013 +0200 @@ -34,10 +34,13 @@ import org.tmatesoft.hg.core.HgStatus.Kind; import org.tmatesoft.hg.core.HgStatusCommand; import org.tmatesoft.hg.core.Nodeid; +import org.tmatesoft.hg.core.SessionContext; import org.tmatesoft.hg.internal.ByteArrayChannel; +import org.tmatesoft.hg.internal.COWTransaction; import org.tmatesoft.hg.internal.CommitFacility; import org.tmatesoft.hg.internal.FileContentSupplier; import org.tmatesoft.hg.internal.Internals; +import org.tmatesoft.hg.internal.Transaction; import org.tmatesoft.hg.repo.HgDataFile; import org.tmatesoft.hg.repo.HgLookup; import org.tmatesoft.hg.repo.HgRepository; @@ -55,7 +58,14 @@ @Rule public ErrorCollectorExt errorCollector = new ErrorCollectorExt(); - + + private final Transaction.Factory trFactory = new COWTransaction.Factory(); +// { +// public Transaction create(Source ctxSource) { +// return new Transaction.NoRollback(); +// } +// }; + @Test public void testCommitToNonEmpty() throws Exception { File repoLoc = RepoUtils.initEmptyTempRepo("test-commit2non-empty"); @@ -68,7 +78,9 @@ // just changed endings are in the patch! HgDataFile df = hgRepo.getFileNode("file1"); cf.add(df, new ByteArraySupplier("hello\nworld".getBytes())); - Nodeid secondRev = cf.commit("SECOND"); + Transaction tr = newTransaction(hgRepo); + Nodeid secondRev = cf.commit("SECOND", tr); + tr.commit(); // List commits = new HgLogCommand(hgRepo).execute(); errorCollector.assertEquals(2, commits.size()); @@ -96,7 +108,9 @@ final byte[] initialContent = "hello\nworld".getBytes(); cf.add(df, new ByteArraySupplier(initialContent)); String comment = "commit 1"; - Nodeid c1Rev = cf.commit(comment); + Transaction tr = newTransaction(hgRepo); + Nodeid c1Rev = cf.commit(comment, tr); + tr.commit(); List commits = new HgLogCommand(hgRepo).execute(); errorCollector.assertEquals(1, commits.size()); HgChangeset c1 = commits.get(0); @@ -130,7 +144,9 @@ FileContentSupplier contentProvider = new FileContentSupplier(fileD); cf.add(dfD, contentProvider); cf.branch("branch1"); - Nodeid commitRev1 = cf.commit("FIRST"); + Transaction tr = newTransaction(hgRepo); + Nodeid commitRev1 = cf.commit("FIRST", tr); + tr.commit(); contentProvider.done(); // List commits = new HgLogCommand(hgRepo).range(parentCsetRevIndex+1, TIP).execute(); @@ -158,7 +174,9 @@ FileContentSupplier contentProvider = new FileContentSupplier(new File(repoLoc, "xx")); cf.add(hgRepo.getFileNode("xx"), contentProvider); cf.forget(hgRepo.getFileNode("d")); - Nodeid commitRev = cf.commit("Commit with add/remove cmd"); + Transaction tr = newTransaction(hgRepo); + Nodeid commitRev = cf.commit("Commit with add/remove cmd", tr); + tr.commit(); contentProvider.done(); // List commits = new HgLogCommand(hgRepo).changeset(commitRev).execute(); @@ -191,20 +209,22 @@ FileContentSupplier contentProvider = new FileContentSupplier(fileD); cf.add(dfD, contentProvider); cf.branch("branch1"); - Nodeid commitRev1 = cf.commit("FIRST"); + Transaction tr = newTransaction(hgRepo); + Nodeid commitRev1 = cf.commit("FIRST", tr); contentProvider.done(); // RepoUtils.modifyFileAppend(fileD, " 2 \n"); cf.add(dfD, contentProvider = new FileContentSupplier(fileD)); cf.branch("branch2"); - Nodeid commitRev2 = cf.commit("SECOND"); + Nodeid commitRev2 = cf.commit("SECOND", tr); contentProvider.done(); // RepoUtils.modifyFileAppend(fileD, " 2 \n"); cf.add(dfD, contentProvider = new FileContentSupplier(fileD)); cf.branch(DEFAULT_BRANCH_NAME); - Nodeid commitRev3 = cf.commit("THIRD"); + Nodeid commitRev3 = cf.commit("THIRD", tr); contentProvider.done(); + tr.commit(); // List commits = new HgLogCommand(hgRepo).range(parentCsetRevIndex+1, TIP).execute(); assertEquals(3, commits.size()); @@ -313,7 +333,9 @@ CommitFacility cf = new CommitFacility(Internals.getInstance(hgRepo), parentCsetRevIndex); cf.add(hgRepo.getFileNode("a"), new FileContentSupplier(new File(repoLoc, "a"))); cf.branch(branch); - Nodeid commit = cf.commit("FIRST"); + Transaction tr = newTransaction(hgRepo); + Nodeid commit = cf.commit("FIRST", tr); + tr.commit(); errorCollector.assertEquals("commit with branch shall update WC", branch, hgRepo.getWorkingCopyBranchName()); ExecHelper eh = new ExecHelper(new OutputParser.Stub(), repoLoc); @@ -331,6 +353,10 @@ verifyRun.run("hg", "verify"); errorCollector.assertEquals("hg verify", 0, verifyRun.getExitValue()); } + + private Transaction newTransaction(SessionContext.Source ctxSource) { + return trFactory.create(ctxSource); + } public static void main(String[] args) throws Exception { new TestCommit().testCommitToEmpty();