Mercurial > hg4j

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 626:5afc7eedb3dd v1.1rc1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
@since, TODOs. Tests: add 1 sec to deal with fs timestamp granularity on linux
author Artem Tikhomirov <tikhomirov.artem@gmail.com>
date Tue, 21 May 2013 19:30:12 +0200
parents 868b2ffdcd5c
children 12a4f60ea972
rev   line source
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
1 /*
584
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
2 * Copyright (c) 2010-2013 TMate Software Ltd
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
3 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
4 * This program is free software; you can redistribute it and/or modify
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
5 * it under the terms of the GNU General Public License as published by
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
6 * the Free Software Foundation; version 2 of the License.
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
7 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
8 * This program is distributed in the hope that it will be useful,
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
11 * GNU General Public License for more details.
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
12 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
13 * For information on how to redistribute this software under
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
14 * the terms of a license other than GNU General Public License
102
a3a2e5deb320 Updated contact address to support@hg4j.com
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 74
diff changeset
15 * contact TMate Software at support@hg4j.com
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
16 */
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
17 package org.tmatesoft.hg.internal;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
18
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
19 import static org.tmatesoft.hg.util.LogFacility.Severity.Error;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
20 import static org.tmatesoft.hg.util.LogFacility.Severity.Warn;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
21
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
22 import java.io.File;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
23 import java.io.FileInputStream;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
24 import java.io.FileOutputStream;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
25 import java.io.IOException;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
26 import java.nio.ByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
27 import java.nio.MappedByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
28 import java.nio.channels.FileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
29
617
65c01508f002 Rollback support for commands that modify repository. Strategy to keep complete copy of a file being changed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 608
diff changeset
30 import org.tmatesoft.hg.core.HgIOException;
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
31 import org.tmatesoft.hg.core.SessionContext;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
32 import org.tmatesoft.hg.util.LogFacility;
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
33
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
34 /**
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
35 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
36 * @author Artem Tikhomirov
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
37 * @author TMate Software Ltd.
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
38 */
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
39 public class DataAccessProvider {
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
40 /**
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
41 * Boundary to start using file memory mapping instead of regular file access, in bytes.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
42 * Set to 0 to indicate mapping files into memory shall not be used.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
43 * If set to -1, file of any size would be mapped in memory.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
44 */
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
45 public static final String CFG_PROPERTY_MAPIO_LIMIT = "hg4j.dap.mapio_limit";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
46 public static final String CFG_PROPERTY_MAPIO_BUFFER_SIZE = "hg4j.dap.mapio_buffer";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
47 public static final String CFG_PROPERTY_FILE_BUFFER_SIZE = "hg4j.dap.file_buffer";
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
48
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
49 private static final int DEFAULT_MAPIO_LIMIT = 100 * 1024; // 100 kB
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
50 private static final int DEFAULT_FILE_BUFFER = 8 * 1024; // 8 kB
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
51 private static final int DEFAULT_MAPIO_BUFFER = DEFAULT_MAPIO_LIMIT; // same as default boundary
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
52
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
53 private final int mapioMagicBoundary;
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
54 private final int bufferSize, mapioBufSize;
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
55 private final SessionContext context;
584
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
56
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
57 public DataAccessProvider(SessionContext ctx) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
58 context = ctx;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
59 PropertyMarshal pm = new PropertyMarshal(ctx);
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
60 mapioMagicBoundary = mapioBoundaryValue(pm.getInt(CFG_PROPERTY_MAPIO_LIMIT, DEFAULT_MAPIO_LIMIT));
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
61 bufferSize = pm.getInt(CFG_PROPERTY_FILE_BUFFER_SIZE, DEFAULT_FILE_BUFFER);
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
62 mapioBufSize = pm.getInt(CFG_PROPERTY_MAPIO_BUFFER_SIZE, DEFAULT_MAPIO_BUFFER);
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
63 }
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
64
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
65 public DataAccessProvider(SessionContext ctx, int mapioBoundary, int regularBufferSize, int mapioBufferSize) {
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
66 context = ctx;
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
67 mapioMagicBoundary = mapioBoundaryValue(mapioBoundary);
27
b0a15cefdfd6 Cons args instead of fixed consts
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 26
diff changeset
68 bufferSize = regularBufferSize;
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
69 mapioBufSize = mapioBufferSize;
584
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
70 }
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 538
diff changeset
71
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
72 // ensure contract of CFG_PROPERTY_MAPIO_LIMIT, for mapioBoundary == 0 use MAX_VALUE so that no file is memmap-ed
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
73 private static int mapioBoundaryValue(int mapioBoundary) {
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
74 return mapioBoundary == 0 ? Integer.MAX_VALUE : mapioBoundary;
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
75 }
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
76
606
5daa42067e7c Avoid mmap files when only few bytes are to be read
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 584
diff changeset
77 public DataAccess createReader(File f, boolean shortRead) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
78 if (!f.exists()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
79 return new DataAccess();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
80 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
81 try {
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
82 FileInputStream fis = new FileInputStream(f);
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
83 long flen = f.length();
606
5daa42067e7c Avoid mmap files when only few bytes are to be read
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 584
diff changeset
84 if (!shortRead && flen > mapioMagicBoundary) {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
85 // TESTS: bufLen of 1024 was used to test MemMapFileAccess
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
86 return new MemoryMapFileAccess(fis, flen, mapioBufSize, context.getLog());
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
87 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
88 // XXX once implementation is more or less stable,
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
89 // may want to try ByteBuffer.allocateDirect() to see
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
90 // if there's any performance gain.
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
91 boolean useDirectBuffer = false; // XXX might be another config option
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
92 // TESTS: bufferSize of 100 was used to check buffer underflow states when readBytes reads chunks bigger than bufSize
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
93 return new FileAccess(fis, flen, bufferSize, useDirectBuffer, context.getLog());
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
94 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
95 } catch (IOException ex) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
96 // unlikely to happen, we've made sure file exists.
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
97 context.getLog().dump(getClass(), Error, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
98 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
99 return new DataAccess(); // non-null, empty.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
100 }
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
101
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
102 public DataSerializer createWriter(final Transaction tr, File f, boolean createNewIfDoesntExist) {
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
103 if (!f.exists() && !createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
104 return new DataSerializer();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
105 }
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
106 // TODO invert RevlogStreamWriter to send DataSource here instead of grabbing DataSerializer
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
107 // to control the moment transaction gets into play and whether it fails or not
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
108 return new TransactionAwareFileSerializer(tr, f);
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
109 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
110
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
111 private static class MemoryMapFileAccess extends DataAccess {
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
112 private FileInputStream fileStream;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
113 private FileChannel fileChannel;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
114 private long position = 0; // always points to buffer's absolute position in the file
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
115 private MappedByteBuffer buffer;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
116 private final long size;
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
117 private final int memBufferSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
118 private final LogFacility logFacility;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
119
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
120 public MemoryMapFileAccess(FileInputStream fis, long channelSize, int bufferSize, LogFacility log) {
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
121 fileStream = fis;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
122 fileChannel = fis.getChannel();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
123 size = channelSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
124 logFacility = log;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
125 memBufferSize = bufferSize > channelSize ? (int) channelSize : bufferSize; // no reason to waste memory more than there's data
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
126 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
127
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
128 @Override
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
129 public boolean isEmpty() {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
130 return position + (buffer == null ? 0 : buffer.position()) >= size;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
131 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
132
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
133 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
134 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
135 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
136 return this;
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
137 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
138
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
139 @Override
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
140 public int length() {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
141 return Internals.ltoi(longLength());
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
142 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
143
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
144 @Override
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
145 public long longLength() {
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
146 return size;
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
147 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
148
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
149 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
150 public void longSeek(long offset) {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
151 assert offset >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
152 // offset may not necessarily be further than current position in the file (e.g. rewind)
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
153 if (buffer != null && /*offset is within buffer*/ offset >= position && (offset - position) < buffer.limit()) {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
154 // cast is ok according to check above
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
155 buffer.position(Internals.ltoi(offset - position));
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
156 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
157 position = offset;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
158 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
159 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
160 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
161
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
162 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
163 public void seek(int offset) {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
164 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
165 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
166
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
167 @Override
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
168 public void skip(int bytes) throws IOException {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
169 assert bytes >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
170 if (buffer == null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
171 position += bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
172 return;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
173 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
174 if (buffer.remaining() > bytes) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
175 buffer.position(buffer.position() + bytes);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
176 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
177 position += buffer.position() + bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
178 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
179 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
180 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
181
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
182 private void fill() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
183 if (buffer != null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
184 position += buffer.position();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
185 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
186 long left = size - position;
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
187 for (int i = 0; i < 3; i++) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
188 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
189 buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, position, left < memBufferSize ? left : memBufferSize);
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
190 return;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
191 } catch (IOException ex) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
192 if (i == 2) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
193 throw ex;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
194 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
195 if (i == 0) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
196 // if first attempt failed, try to free some virtual memory, see Issue 30 for details
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
197 logFacility.dump(getClass(), Warn, ex, "Memory-map failed, gonna try gc() to free virtual memory");
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
198 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
199 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
200 buffer = null;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
201 System.gc();
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
202 Thread.sleep((1+i) * 1000);
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
203 } catch (Throwable t) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
204 logFacility.dump(getClass(), Error, t, "Bad luck");
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
205 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
206 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
207 }
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
208 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
209
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
210 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
211 public void readBytes(byte[] buf, int offset, int length) throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
212 if (buffer == null || !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
213 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
214 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
215 // XXX in fact, we may try to create a MappedByteBuffer of exactly length size here, and read right away
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
216 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
217 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
218 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
219 throw new IOException();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
220 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
221 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
222 buffer.get(buf, offset, length);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
223 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
224 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
225 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
226 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
227 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
228 length -= tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
229 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
230 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
231
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
232 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
233 public byte readByte() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
234 if (buffer == null || !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
235 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
236 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
237 if (buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
238 return buffer.get();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
239 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
240 throw new IOException();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
241 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
242
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
243 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
244 public void done() {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
245 buffer = null;
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
246 if (fileStream != null) {
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
247 new FileUtils(logFacility).closeQuietly(fileStream);
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
248 fileStream = null;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
249 fileChannel = null; // channel is closed together with stream
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
250 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
251 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
252 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
253
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
254 // (almost) regular file access - FileChannel and buffers.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
255 private static class FileAccess extends DataAccess {
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
256 private FileInputStream fileStream;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
257 private FileChannel fileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
258 private ByteBuffer buffer;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
259 private long bufferStartInFile = 0; // offset of this.buffer in the file.
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
260 private final long size;
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
261 private final LogFacility logFacility;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
262
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
263 public FileAccess(FileInputStream fis, long channelSize, int bufferSizeHint, boolean useDirect, LogFacility log) {
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
264 fileStream = fis;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
265 fileChannel = fis.getChannel();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
266 size = channelSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
267 logFacility = log;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
268 final int capacity = size < bufferSizeHint ? (int) size : bufferSizeHint;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
269 buffer = useDirect ? ByteBuffer.allocateDirect(capacity) : ByteBuffer.allocate(capacity);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
270 buffer.flip(); // or .limit(0) to indicate it's empty
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
271 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
272
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
273 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
274 public boolean isEmpty() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
275 return bufferStartInFile + buffer.position() >= size;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
276 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
277
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
278 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
279 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
280 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
281 return this;
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
282 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
283
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
284 @Override
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
285 public int length() {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
286 return Internals.ltoi(longLength());
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
287 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
288
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
289 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
290 public long longLength() {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
291 return size;
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
292 }
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
293
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
294 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
295 public void longSeek(long offset) throws IOException {
23
6f9aca1a97be Severe defect in buffer wrap on seek
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 10
diff changeset
296 if (offset > size) {
404
31a719b9f95e Provide more detailed information on erroneous file operation
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 338
diff changeset
297 throw new IllegalArgumentException(String.format("Can't seek to %d for the file of size %d (buffer start:%d)", offset, size, bufferStartInFile));
23
6f9aca1a97be Severe defect in buffer wrap on seek
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 10
diff changeset
298 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
299 if (offset < bufferStartInFile + buffer.limit() && offset >= bufferStartInFile) {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
300 // cast to int is safe, we've checked we fit into buffer
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
301 buffer.position(Internals.ltoi(offset - bufferStartInFile));
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
302 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
303 // out of current buffer, invalidate it (force re-read)
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
304 // XXX or ever re-read it right away?
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
305 bufferStartInFile = offset;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
306 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
307 buffer.limit(0); // or .flip() to indicate we switch to reading
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
308 fileChannel.position(offset);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
309 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
310 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
311
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
312 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
313 public void seek(int offset) throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
314 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
315 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
316
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
317 @Override
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
318 public void skip(int bytes) throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
319 final int newPos = buffer.position() + bytes;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
320 if (newPos >= 0 && newPos < buffer.limit()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
321 // no need to move file pointer, just rewind/seek buffer
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
322 buffer.position(newPos);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
323 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
324 //
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
325 longSeek(bufferStartInFile + newPos);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
326 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
327 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
328
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
329 private boolean fill() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
330 if (!buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
331 bufferStartInFile += buffer.limit();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
332 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
333 if (bufferStartInFile < size) { // just in case there'd be any exception on EOF, not -1
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
334 fileChannel.read(buffer);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
335 // may return -1 when EOF, but empty will reflect this, hence no explicit support here
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
336 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
337 buffer.flip();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
338 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
339 return buffer.hasRemaining();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
340 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
341
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
342 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
343 public void readBytes(byte[] buf, int offset, int length) throws IOException {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
344 if (!buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
345 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
346 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
347 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
348 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
349 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
350 throw new IOException(); // shall not happen provided stream contains expected data and no attempts to read past isEmpty() == true are made.
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
351 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
352 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
353 buffer.get(buf, offset, length);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
354 } else {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
355 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
356 fill();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
357 }
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
358 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
359 length -= tail;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
360 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
361 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
362
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
363 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
364 public byte readByte() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
365 if (buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
366 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
367 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
368 if (fill()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
369 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
370 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
371 throw new IOException();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
372 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
373
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
374 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
375 public void done() {
619
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
376 buffer = null;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
377 if (fileStream != null) {
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
378 new FileUtils(logFacility).closeQuietly(fileStream);
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 618
diff changeset
379 fileStream = null;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
380 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
381 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
382 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
383 }
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
384
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
385 /**
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
386 * Appends serialized changes to the end of the file
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
387 */
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
388 private static class TransactionAwareFileSerializer extends DataSerializer {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
389
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
390 private final Transaction transaction;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
391 private final File file;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
392 private FileOutputStream fos;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
393 private File transactionFile;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
394 private boolean writeFailed = false;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
395
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
396 public TransactionAwareFileSerializer(Transaction tr, File f) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
397 transaction = tr;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
398 file = f;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
399 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
400
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
401 @Override
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
402 public void write(byte[] data, int offset, int length) throws HgIOException {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
403 try {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
404 if (fos == null) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
405 transactionFile = transaction.prepare(file);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
406 fos = new FileOutputStream(transactionFile, true);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
407 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
408 fos.write(data, offset, length);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
409 fos.flush();
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
410 } catch (IOException ex) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
411 writeFailed = true;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
412 transaction.failure(transactionFile, ex);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
413 throw new HgIOException("Write failure", ex, transactionFile);
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
414 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
415 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
416
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
417 @Override
618
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
418 public void done() throws HgIOException {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
419 if (fos != null) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
420 assert transactionFile != null;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
421 try {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
422 fos.close();
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
423 if (!writeFailed) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
424 // XXX, Transaction#done() assumes there's no error , but perhaps it's easier to
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
425 // rely on #failure(), and call #done() always (or change #done() to #success()
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
426 transaction.done(transactionFile);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
427 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
428 fos = null;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
429 } catch (IOException ex) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
430 if (!writeFailed) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
431 // do not eclipse original exception
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
432 transaction.failure(transactionFile, ex);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
433 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
434 throw new HgIOException("Write failure", ex, transactionFile);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 617
diff changeset
435 }
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
436 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
437 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
438 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
439 }