Mercurial > hg4j

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 618:7c0d2ce340b8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Refactor approach how content finds it way down to a commit revision
author Artem Tikhomirov <tikhomirov.artem@gmail.com>
date Thu, 16 May 2013 19:46:13 +0200
parents 65c01508f002
children 868b2ffdcd5c
rev   line source
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
1 /*
584
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
2 * Copyright (c) 2010-2013 TMate Software Ltd
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
3 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
4 * This program is free software; you can redistribute it and/or modify
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
5 * it under the terms of the GNU General Public License as published by
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
6 * the Free Software Foundation; version 2 of the License.
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
7 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
8 * This program is distributed in the hope that it will be useful,
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
11 * GNU General Public License for more details.
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
12 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
13 * For information on how to redistribute this software under
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
14 * the terms of a license other than GNU General Public License
102
a3a2e5deb320 Updated contact address to support@hg4j.com
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 74
diff changeset
15 * contact TMate Software at support@hg4j.com
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
16 */
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
17 package org.tmatesoft.hg.internal;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
18
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
19 import static org.tmatesoft.hg.util.LogFacility.Severity.Error;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
20 import static org.tmatesoft.hg.util.LogFacility.Severity.Warn;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
21
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
22 import java.io.File;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
23 import java.io.FileInputStream;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
24 import java.io.FileOutputStream;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
25 import java.io.IOException;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
26 import java.nio.ByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
27 import java.nio.MappedByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
28 import java.nio.channels.FileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
29
617
65c01508f002 Rollback support for commands that modify repository. Strategy to keep complete copy of a file being changed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 608
diff changeset
30 import org.tmatesoft.hg.core.HgIOException;
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
31 import org.tmatesoft.hg.core.SessionContext;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
32 import org.tmatesoft.hg.util.LogFacility;
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
33
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
34 /**
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
35 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
36 * @author Artem Tikhomirov
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
37 * @author TMate Software Ltd.
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
38 */
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
39 public class DataAccessProvider {
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
40 /**
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
41 * Boundary to start using file memory mapping instead of regular file access, in bytes.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
42 * Set to 0 to indicate mapping files into memory shall not be used.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
43 * If set to -1, file of any size would be mapped in memory.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
44 */
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
45 public static final String CFG_PROPERTY_MAPIO_LIMIT = "hg4j.dap.mapio_limit";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
46 public static final String CFG_PROPERTY_MAPIO_BUFFER_SIZE = "hg4j.dap.mapio_buffer";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
47 public static final String CFG_PROPERTY_FILE_BUFFER_SIZE = "hg4j.dap.file_buffer";
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
48
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
49 private static final int DEFAULT_MAPIO_LIMIT = 100 * 1024; // 100 kB
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
50 private static final int DEFAULT_FILE_BUFFER = 8 * 1024; // 8 kB
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
51 private static final int DEFAULT_MAPIO_BUFFER = DEFAULT_MAPIO_LIMIT; // same as default boundary
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
52
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
53 private final int mapioMagicBoundary;
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
54 private final int bufferSize, mapioBufSize;
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
55 private final SessionContext context;
584
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
56
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
57 public DataAccessProvider(SessionContext ctx) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
58 context = ctx;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
59 PropertyMarshal pm = new PropertyMarshal(ctx);
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
60 mapioMagicBoundary = mapioBoundaryValue(pm.getInt(CFG_PROPERTY_MAPIO_LIMIT, DEFAULT_MAPIO_LIMIT));
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
61 bufferSize = pm.getInt(CFG_PROPERTY_FILE_BUFFER_SIZE, DEFAULT_FILE_BUFFER);
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
62 mapioBufSize = pm.getInt(CFG_PROPERTY_MAPIO_BUFFER_SIZE, DEFAULT_MAPIO_BUFFER);
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
63 }
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
64
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
65 public DataAccessProvider(SessionContext ctx, int mapioBoundary, int regularBufferSize, int mapioBufferSize) {
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
66 context = ctx;
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
67 mapioMagicBoundary = mapioBoundaryValue(mapioBoundary);
27
b0a15cefdfd6 Cons args instead of fixed consts
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 26
diff changeset
68 bufferSize = regularBufferSize;
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
69 mapioBufSize = mapioBufferSize;
584
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
70 }
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
71
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
72 // ensure contract of CFG_PROPERTY_MAPIO_LIMIT, for mapioBoundary == 0 use MAX_VALUE so that no file is memmap-ed
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
73 private static int mapioBoundaryValue(int mapioBoundary) {
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
74 return mapioBoundary == 0 ? Integer.MAX_VALUE : mapioBoundary;
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
75 }
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
76
606
5daa42067e7c Avoid mmap files when only few bytes are to be read
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 584
diff changeset
77 public DataAccess createReader(File f, boolean shortRead) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
78 if (!f.exists()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
79 return new DataAccess();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
80 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
81 try {
617
65c01508f002 Rollback support for commands that modify repository. Strategy to keep complete copy of a file being changed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 608
diff changeset
82 FileChannel fc = new FileInputStream(f).getChannel(); // FIXME SHALL CLOSE FIS, not only channel
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
83 long flen = fc.size();
606
5daa42067e7c Avoid mmap files when only few bytes are to be read
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 584
diff changeset
84 if (!shortRead && flen > mapioMagicBoundary) {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
85 // TESTS: bufLen of 1024 was used to test MemMapFileAccess
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
86 return new MemoryMapFileAccess(fc, flen, mapioBufSize, context.getLog());
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
87 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
88 // XXX once implementation is more or less stable,
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
89 // may want to try ByteBuffer.allocateDirect() to see
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
90 // if there's any performance gain.
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
91 boolean useDirectBuffer = false; // XXX might be another config option
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
92 // TESTS: bufferSize of 100 was used to check buffer underflow states when readBytes reads chunks bigger than bufSize
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
93 return new FileAccess(fc, flen, bufferSize, useDirectBuffer, context.getLog());
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
94 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
95 } catch (IOException ex) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
96 // unlikely to happen, we've made sure file exists.
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
97 context.getLog().dump(getClass(), Error, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
98 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
99 return new DataAccess(); // non-null, empty.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
100 }
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
101
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
102 public DataSerializer createWriter(final Transaction tr, File f, boolean createNewIfDoesntExist) {
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
103 if (!f.exists() && !createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
104 return new DataSerializer();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
105 }
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
106 // TODO invert RevlogStreamWriter to send DataSource here instead of grabbing DataSerializer
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
107 // to control the moment transaction gets into play and whether it fails or not
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
108 return new TransactionAwareFileSerializer(tr, f);
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
109 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
110
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
111 private static class MemoryMapFileAccess extends DataAccess {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
112 private FileChannel fileChannel;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
113 private long position = 0; // always points to buffer's absolute position in the file
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
114 private MappedByteBuffer buffer;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
115 private final long size;
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
116 private final int memBufferSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
117 private final LogFacility logFacility;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
118
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
119 public MemoryMapFileAccess(FileChannel fc, long channelSize, int bufferSize, LogFacility log) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
120 fileChannel = fc;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
121 size = channelSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
122 logFacility = log;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
123 memBufferSize = bufferSize > channelSize ? (int) channelSize : bufferSize; // no reason to waste memory more than there's data
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
124 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
125
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
126 @Override
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
127 public boolean isEmpty() {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
128 return position + (buffer == null ? 0 : buffer.position()) >= size;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
129 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
130
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
131 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
132 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
133 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
134 return this;
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
135 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
136
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
137 @Override
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
138 public int length() {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
139 return Internals.ltoi(longLength());
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
140 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
141
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
142 @Override
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
143 public long longLength() {
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
144 return size;
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
145 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
146
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
147 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
148 public void longSeek(long offset) {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
149 assert offset >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
150 // offset may not necessarily be further than current position in the file (e.g. rewind)
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
151 if (buffer != null && /*offset is within buffer*/ offset >= position && (offset - position) < buffer.limit()) {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
152 // cast is ok according to check above
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
153 buffer.position(Internals.ltoi(offset - position));
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
154 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
155 position = offset;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
156 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
157 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
158 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
159
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
160 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
161 public void seek(int offset) {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
162 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
163 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
164
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
165 @Override
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
166 public void skip(int bytes) throws IOException {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
167 assert bytes >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
168 if (buffer == null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
169 position += bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
170 return;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
171 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
172 if (buffer.remaining() > bytes) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
173 buffer.position(buffer.position() + bytes);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
174 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
175 position += buffer.position() + bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
176 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
177 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
178 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
179
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
180 private void fill() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
181 if (buffer != null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
182 position += buffer.position();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
183 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
184 long left = size - position;
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
185 for (int i = 0; i < 3; i++) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
186 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
187 buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, position, left < memBufferSize ? left : memBufferSize);
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
188 return;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
189 } catch (IOException ex) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
190 if (i == 2) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
191 throw ex;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
192 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
193 if (i == 0) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
194 // if first attempt failed, try to free some virtual memory, see Issue 30 for details
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
195 logFacility.dump(getClass(), Warn, ex, "Memory-map failed, gonna try gc() to free virtual memory");
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
196 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
197 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
198 buffer = null;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
199 System.gc();
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
200 Thread.sleep((1+i) * 1000);
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
201 } catch (Throwable t) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
202 logFacility.dump(getClass(), Error, t, "Bad luck");
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
203 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
204 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
205 }
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
206 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
207
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
208 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
209 public void readBytes(byte[] buf, int offset, int length) throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
210 if (buffer == null || !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
211 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
212 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
213 // XXX in fact, we may try to create a MappedByteBuffer of exactly length size here, and read right away
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
214 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
215 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
216 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
217 throw new IOException();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
218 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
219 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
220 buffer.get(buf, offset, length);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
221 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
222 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
223 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
224 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
225 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
226 length -= tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
227 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
228 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
229
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
230 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
231 public byte readByte() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
232 if (buffer == null || !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
233 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
234 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
235 if (buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
236 return buffer.get();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
237 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
238 throw new IOException();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
239 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
240
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
241 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
242 public void done() {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
243 buffer = null;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
244 if (fileChannel != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
245 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
246 fileChannel.close();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
247 } catch (IOException ex) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
248 logFacility.dump(getClass(), Warn, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
249 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
250 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
251 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
252 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
253 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
254
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
255 // (almost) regular file access - FileChannel and buffers.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
256 private static class FileAccess extends DataAccess {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
257 private FileChannel fileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
258 private ByteBuffer buffer;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
259 private long bufferStartInFile = 0; // offset of this.buffer in the file.
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
260 private final long size;
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
261 private final LogFacility logFacility;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
262
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
263 public FileAccess(FileChannel fc, long channelSize, int bufferSizeHint, boolean useDirect, LogFacility log) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
264 fileChannel = fc;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
265 size = channelSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
266 logFacility = log;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
267 final int capacity = size < bufferSizeHint ? (int) size : bufferSizeHint;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
268 buffer = useDirect ? ByteBuffer.allocateDirect(capacity) : ByteBuffer.allocate(capacity);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
269 buffer.flip(); // or .limit(0) to indicate it's empty
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
270 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
271
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
272 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
273 public boolean isEmpty() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
274 return bufferStartInFile + buffer.position() >= size;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
275 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
276
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
277 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
278 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
279 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
280 return this;
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
281 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
282
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
283 @Override
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
284 public int length() {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
285 return Internals.ltoi(longLength());
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
286 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
287
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
288 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
289 public long longLength() {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
290 return size;
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
291 }
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
292
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
293 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
294 public void longSeek(long offset) throws IOException {
23
6f9aca1a97be Severe defect in buffer wrap on seek
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 10
diff changeset
295 if (offset > size) {
404
31a719b9f95e Provide more detailed information on erroneous file operation
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 338
diff changeset
296 throw new IllegalArgumentException(String.format("Can't seek to %d for the file of size %d (buffer start:%d)", offset, size, bufferStartInFile));
23
6f9aca1a97be Severe defect in buffer wrap on seek
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 10
diff changeset
297 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
298 if (offset < bufferStartInFile + buffer.limit() && offset >= bufferStartInFile) {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
299 // cast to int is safe, we've checked we fit into buffer
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
300 buffer.position(Internals.ltoi(offset - bufferStartInFile));
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
301 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
302 // out of current buffer, invalidate it (force re-read)
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
303 // XXX or ever re-read it right away?
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
304 bufferStartInFile = offset;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
305 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
306 buffer.limit(0); // or .flip() to indicate we switch to reading
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
307 fileChannel.position(offset);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
308 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
309 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
310
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
311 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
312 public void seek(int offset) throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
313 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
314 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
315
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
316 @Override
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
317 public void skip(int bytes) throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
318 final int newPos = buffer.position() + bytes;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
319 if (newPos >= 0 && newPos < buffer.limit()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
320 // no need to move file pointer, just rewind/seek buffer
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
321 buffer.position(newPos);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
322 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
323 //
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
324 longSeek(bufferStartInFile + newPos);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
325 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
326 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
327
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
328 private boolean fill() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
329 if (!buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
330 bufferStartInFile += buffer.limit();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
331 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
332 if (bufferStartInFile < size) { // just in case there'd be any exception on EOF, not -1
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
333 fileChannel.read(buffer);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
334 // may return -1 when EOF, but empty will reflect this, hence no explicit support here
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
335 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
336 buffer.flip();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
337 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
338 return buffer.hasRemaining();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
339 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
340
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
341 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
342 public void readBytes(byte[] buf, int offset, int length) throws IOException {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
343 if (!buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
344 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
345 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
346 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
347 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
348 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
349 throw new IOException(); // shall not happen provided stream contains expected data and no attempts to read past isEmpty() == true are made.
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
350 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
351 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
352 buffer.get(buf, offset, length);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
353 } else {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
354 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
355 fill();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
356 }
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
357 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
358 length -= tail;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
359 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
360 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
361
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
362 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
363 public byte readByte() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
364 if (buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
365 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
366 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
367 if (fill()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
368 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
369 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
370 throw new IOException();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
371 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
372
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
373 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
374 public void done() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
375 if (buffer != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
376 buffer = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
377 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
378 if (fileChannel != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
379 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
380 fileChannel.close();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
381 } catch (IOException ex) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
382 logFacility.dump(getClass(), Warn, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
383 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
384 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
385 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
386 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
387 }
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
388
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
389 /**
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
390 * Appends serialized changes to the end of the file
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
391 */
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
392 private static class TransactionAwareFileSerializer extends DataSerializer {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
393
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
394 private final Transaction transaction;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
395 private final File file;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
396 private FileOutputStream fos;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
397 private File transactionFile;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
398 private boolean writeFailed = false;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
399
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
400 public TransactionAwareFileSerializer(Transaction tr, File f) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
401 transaction = tr;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
402 file = f;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
403 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
404
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
405 @Override
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
406 public void write(byte[] data, int offset, int length) throws HgIOException {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
407 try {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
408 if (fos == null) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
409 transactionFile = transaction.prepare(file);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
410 fos = new FileOutputStream(transactionFile, true);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
411 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
412 fos.write(data, offset, length);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
413 fos.flush();
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
414 } catch (IOException ex) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
415 writeFailed = true;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
416 transaction.failure(transactionFile, ex);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
417 throw new HgIOException("Write failure", ex, transactionFile);
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
418 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
419 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
420
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
421 @Override
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
422 public void done() throws HgIOException {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
423 if (fos != null) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
424 assert transactionFile != null;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
425 try {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
426 fos.close();
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
427 if (!writeFailed) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
428 // XXX, Transaction#done() assumes there's no error , but perhaps it's easier to
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
429 // rely on #failure(), and call #done() always (or change #done() to #success()
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
430 transaction.done(transactionFile);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
431 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
432 fos = null;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
433 } catch (IOException ex) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
434 if (!writeFailed) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
435 // do not eclipse original exception
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
436 transaction.failure(transactionFile, ex);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
437 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
438 throw new HgIOException("Write failure", ex, transactionFile);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
439 }
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
440 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
441 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
442 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
443 }