Mercurial > hg4j

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 575:8bf184c9d733

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue 43: poor performance with InflaterDataAccess. Phase 1: test existing code, fix defects found
author Artem Tikhomirov <tikhomirov.artem@gmail.com>
date Tue, 16 Apr 2013 16:59:59 +0200
parents dd4f6311af52
children ed243b668502
rev   line source
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
1 /*
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
2 * Copyright (c) 2010-2012 TMate Software Ltd
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
3 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
4 * This program is free software; you can redistribute it and/or modify
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
5 * it under the terms of the GNU General Public License as published by
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
6 * the Free Software Foundation; version 2 of the License.
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
7 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
8 * This program is distributed in the hope that it will be useful,
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
11 * GNU General Public License for more details.
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
12 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
13 * For information on how to redistribute this software under
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
14 * the terms of a license other than GNU General Public License
102
a3a2e5deb320 Updated contact address to support@hg4j.com
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 74
diff changeset
15 * contact TMate Software at support@hg4j.com
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
16 */
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
17 package org.tmatesoft.hg.internal;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
18
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
19 import static org.tmatesoft.hg.util.LogFacility.Severity.Error;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
20 import static org.tmatesoft.hg.util.LogFacility.Severity.Warn;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
21
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
22 import java.io.File;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
23 import java.io.FileInputStream;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
24 import java.io.FileNotFoundException;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
25 import java.io.FileOutputStream;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
26 import java.io.IOException;
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
27 import java.io.OutputStream;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
28 import java.nio.ByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
29 import java.nio.MappedByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
30 import java.nio.channels.FileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
31
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
32 import org.tmatesoft.hg.core.SessionContext;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
33 import org.tmatesoft.hg.util.LogFacility;
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
34
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
35 /**
74
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
36 *
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
37 * @author Artem Tikhomirov
6f1b88693d48 Complete refactoring to org.tmatesoft
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
38 * @author TMate Software Ltd.
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
39 */
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
40 public class DataAccessProvider {
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
41 /**
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
42 * Boundary to start using file memory mapping instead of regular file access, in bytes.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
43 * Set to 0 to indicate mapping files into memory shall not be used.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
44 * If set to -1, file of any size would be mapped in memory.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
45 */
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
46 public static final String CFG_PROPERTY_MAPIO_LIMIT = "hg4j.dap.mapio_limit";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
47 public static final String CFG_PROPERTY_MAPIO_BUFFER_SIZE = "hg4j.dap.mapio_buffer";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
48 public static final String CFG_PROPERTY_FILE_BUFFER_SIZE = "hg4j.dap.file_buffer";
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
49
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
50 private static final int DEFAULT_MAPIO_LIMIT = 100 * 1024; // 100 kB
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
51 private static final int DEFAULT_FILE_BUFFER = 8 * 1024; // 8 kB
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
52 private static final int DEFAULT_MAPIO_BUFFER = DEFAULT_MAPIO_LIMIT; // same as default boundary
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
53
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
54 private final int mapioMagicBoundary;
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
55 private final int bufferSize, mapioBufSize;
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
56 private final SessionContext context;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
57
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
58 public DataAccessProvider(SessionContext ctx) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
59 context = ctx;
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
60 PropertyMarshal pm = new PropertyMarshal(ctx);
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
61 mapioMagicBoundary = mapioBoundaryValue(pm.getInt(CFG_PROPERTY_MAPIO_LIMIT, DEFAULT_MAPIO_LIMIT));
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
62 bufferSize = pm.getInt(CFG_PROPERTY_FILE_BUFFER_SIZE, DEFAULT_FILE_BUFFER);
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
63 mapioBufSize = pm.getInt(CFG_PROPERTY_MAPIO_BUFFER_SIZE, DEFAULT_MAPIO_BUFFER);
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
64 }
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
65
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
66 public DataAccessProvider(SessionContext ctx, int mapioBoundary, int regularBufferSize, int mapioBufferSize) {
295
981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 158
diff changeset
67 context = ctx;
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
68 mapioMagicBoundary = mapioBoundaryValue(mapioBoundary);
27
b0a15cefdfd6 Cons args instead of fixed consts
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 26
diff changeset
69 bufferSize = regularBufferSize;
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
70 mapioBufSize = mapioBufferSize;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
71 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
72
494
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
73 // ensure contract of CFG_PROPERTY_MAPIO_LIMIT, for mapioBoundary == 0 use MAX_VALUE so that no file is memmap-ed
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
74 private static int mapioBoundaryValue(int mapioBoundary) {
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
75 return mapioBoundary == 0 ? Integer.MAX_VALUE : mapioBoundary;
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
76 }
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 456
diff changeset
77
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
78 public DataAccess createReader(File f) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
79 if (!f.exists()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
80 return new DataAccess();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
81 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
82 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
83 FileChannel fc = new FileInputStream(f).getChannel();
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
84 long flen = fc.size();
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
85 if (flen > mapioMagicBoundary) {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
86 // TESTS: bufLen of 1024 was used to test MemMapFileAccess
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
87 return new MemoryMapFileAccess(fc, flen, mapioBufSize, context.getLog());
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
88 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
89 // XXX once implementation is more or less stable,
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
90 // may want to try ByteBuffer.allocateDirect() to see
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
91 // if there's any performance gain.
338
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close())
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 295
diff changeset
92 boolean useDirectBuffer = false; // XXX might be another config option
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
93 // TESTS: bufferSize of 100 was used to check buffer underflow states when readBytes reads chunks bigger than bufSize
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
94 return new FileAccess(fc, flen, bufferSize, useDirectBuffer, context.getLog());
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
95 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
96 } catch (IOException ex) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
97 // unlikely to happen, we've made sure file exists.
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
98 context.getLog().dump(getClass(), Error, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
99 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
100 return new DataAccess(); // non-null, empty.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
101 }
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
102
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
103 public DataSerializer createWriter(File f, boolean createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
104 if (!f.exists() && !createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
105 return new DataSerializer();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
106 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
107 try {
538
dd4f6311af52 Commit: first working version
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 534
diff changeset
108 return new StreamDataSerializer(context.getLog(), new FileOutputStream(f, true));
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
109 } catch (final FileNotFoundException ex) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
110 context.getLog().dump(getClass(), Error, ex, null);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
111 return new DataSerializer() {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
112 public void write(byte[] data, int offset, int length) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
113 throw ex;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
114 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
115 };
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
116 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
117 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
118
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
119 private static class MemoryMapFileAccess extends DataAccess {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
120 private FileChannel fileChannel;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
121 private long position = 0; // always points to buffer's absolute position in the file
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
122 private MappedByteBuffer buffer;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
123 private final long size;
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
124 private final int memBufferSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
125 private final LogFacility logFacility;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
126
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
127 public MemoryMapFileAccess(FileChannel fc, long channelSize, int bufferSize, LogFacility log) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
128 fileChannel = fc;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
129 size = channelSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
130 logFacility = log;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
131 memBufferSize = bufferSize > channelSize ? (int) channelSize : bufferSize; // no reason to waste memory more than there's data
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
132 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
133
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
134 @Override
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
135 public boolean isEmpty() {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
136 return position + (buffer == null ? 0 : buffer.position()) >= size;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
137 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
138
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
139 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
140 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
141 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
142 return this;
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
143 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
144
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
145 @Override
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
146 public int length() {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
147 return Internals.ltoi(longLength());
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
148 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
149
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
150 @Override
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
151 public long longLength() {
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
152 return size;
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
153 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
154
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
155 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
156 public void longSeek(long offset) {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
157 assert offset >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
158 // offset may not necessarily be further than current position in the file (e.g. rewind)
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
159 if (buffer != null && /*offset is within buffer*/ offset >= position && (offset - position) < buffer.limit()) {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
160 // cast is ok according to check above
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
161 buffer.position(Internals.ltoi(offset - position));
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
162 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
163 position = offset;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
164 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
165 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
166 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
167
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
168 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
169 public void seek(int offset) {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
170 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
171 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
172
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
173 @Override
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
174 public void skip(int bytes) throws IOException {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
175 assert bytes >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
176 if (buffer == null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
177 position += bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
178 return;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
179 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
180 if (buffer.remaining() > bytes) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
181 buffer.position(buffer.position() + bytes);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
182 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
183 position += buffer.position() + bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
184 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
185 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
186 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
187
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
188 private void fill() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
189 if (buffer != null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
190 position += buffer.position();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
191 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
192 long left = size - position;
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
193 for (int i = 0; i < 3; i++) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
194 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
195 buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, position, left < memBufferSize ? left : memBufferSize);
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
196 return;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
197 } catch (IOException ex) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
198 if (i == 2) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
199 throw ex;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
200 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
201 if (i == 0) {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
202 // if first attempt failed, try to free some virtual memory, see Issue 30 for details
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
203 logFacility.dump(getClass(), Warn, ex, "Memory-map failed, gonna try gc() to free virtual memory");
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
204 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
205 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
206 buffer = null;
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
207 System.gc();
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
208 Thread.sleep((1+i) * 1000);
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
209 } catch (Throwable t) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
210 logFacility.dump(getClass(), Error, t, "Bad luck");
440
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
211 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
212 }
299870249a28 Issue 30: bogus IOException for mmap file on linux
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 425
diff changeset
213 }
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
214 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
215
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
216 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
217 public void readBytes(byte[] buf, int offset, int length) throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
218 if (buffer == null || !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
219 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
220 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
221 // XXX in fact, we may try to create a MappedByteBuffer of exactly length size here, and read right away
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
222 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
223 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
224 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
225 throw new IOException();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
226 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
227 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
228 buffer.get(buf, offset, length);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
229 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
230 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
231 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
232 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
233 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
234 length -= tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
235 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
236 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
237
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
238 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
239 public byte readByte() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
240 if (buffer == null || !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
241 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
242 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
243 if (buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
244 return buffer.get();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
245 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
246 throw new IOException();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
247 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
248
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
249 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
250 public void done() {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
251 buffer = null;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
252 if (fileChannel != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
253 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
254 fileChannel.close();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
255 } catch (IOException ex) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
256 logFacility.dump(getClass(), Warn, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
257 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
258 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
259 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
260 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
261 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
262
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
263 // (almost) regular file access - FileChannel and buffers.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
264 private static class FileAccess extends DataAccess {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
265 private FileChannel fileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
266 private ByteBuffer buffer;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
267 private long bufferStartInFile = 0; // offset of this.buffer in the file.
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
268 private final long size;
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
269 private final LogFacility logFacility;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
270
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
271 public FileAccess(FileChannel fc, long channelSize, int bufferSizeHint, boolean useDirect, LogFacility log) {
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
272 fileChannel = fc;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
273 size = channelSize;
425
48f993aa2f41 FIXMEs: exceptions, javadoc
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 420
diff changeset
274 logFacility = log;
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
275 final int capacity = size < bufferSizeHint ? (int) size : bufferSizeHint;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
276 buffer = useDirect ? ByteBuffer.allocateDirect(capacity) : ByteBuffer.allocate(capacity);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
277 buffer.flip(); // or .limit(0) to indicate it's empty
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
278 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
279
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
280 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
281 public boolean isEmpty() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
282 return bufferStartInFile + buffer.position() >= size;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
283 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
284
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
285 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
286 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
287 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
288 return this;
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
289 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
290
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
291 @Override
158
b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 157
diff changeset
292 public int length() {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
293 return Internals.ltoi(longLength());
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
294 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
295
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
296 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
297 public long longLength() {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
298 return size;
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
299 }
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
300
51
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 27
diff changeset
301 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
302 public void longSeek(long offset) throws IOException {
23
6f9aca1a97be Severe defect in buffer wrap on seek
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 10
diff changeset
303 if (offset > size) {
404
31a719b9f95e Provide more detailed information on erroneous file operation
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 338
diff changeset
304 throw new IllegalArgumentException(String.format("Can't seek to %d for the file of size %d (buffer start:%d)", offset, size, bufferStartInFile));
23
6f9aca1a97be Severe defect in buffer wrap on seek
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 10
diff changeset
305 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
306 if (offset < bufferStartInFile + buffer.limit() && offset >= bufferStartInFile) {
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
307 // cast to int is safe, we've checked we fit into buffer
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
308 buffer.position(Internals.ltoi(offset - bufferStartInFile));
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
309 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
310 // out of current buffer, invalidate it (force re-read)
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
311 // XXX or ever re-read it right away?
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
312 bufferStartInFile = offset;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
313 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
314 buffer.limit(0); // or .flip() to indicate we switch to reading
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
315 fileChannel.position(offset);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
316 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
317 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
318
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
319 @Override
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
320 public void seek(int offset) throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
321 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
322 }
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
323
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
324 @Override
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
325 public void skip(int bytes) throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
326 final int newPos = buffer.position() + bytes;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
327 if (newPos >= 0 && newPos < buffer.limit()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
328 // no need to move file pointer, just rewind/seek buffer
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
329 buffer.position(newPos);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
330 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
331 //
420
6c22bdc0bdfd Respect long offsets in revlogs
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 414
diff changeset
332 longSeek(bufferStartInFile + newPos);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
333 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
334 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
335
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
336 private boolean fill() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
337 if (!buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
338 bufferStartInFile += buffer.limit();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
339 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
340 if (bufferStartInFile < size) { // just in case there'd be any exception on EOF, not -1
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
341 fileChannel.read(buffer);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
342 // may return -1 when EOF, but empty will reflect this, hence no explicit support here
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
343 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
344 buffer.flip();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
345 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
346 return buffer.hasRemaining();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
347 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
348
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
349 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
350 public void readBytes(byte[] buf, int offset, int length) throws IOException {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
351 if (!buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
352 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
353 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
354 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
355 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
356 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
357 throw new IOException(); // shall not happen provided stream contains expected data and no attempts to read past isEmpty() == true are made.
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
358 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
359 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
360 buffer.get(buf, offset, length);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
361 } else {
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
362 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
363 fill();
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
364 }
26
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
365 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 23
diff changeset
366 length -= tail;
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
367 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
368 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
369
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
370 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
371 public byte readByte() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
372 if (buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
373 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
374 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
375 if (fill()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
376 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
377 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
378 throw new IOException();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
379 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
380
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
381 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
382 public void done() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
383 if (buffer != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
384 buffer = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
385 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
386 if (fileChannel != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
387 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
388 fileChannel.close();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
389 } catch (IOException ex) {
456
909306e412e2 Refactor LogFacility and SessionContext, better API for both
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 440
diff changeset
390 logFacility.dump(getClass(), Warn, ex, null);
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
391 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
392 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
393 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
394 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
395 }
534
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
396
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
397 public/*XXX, private, once HgCloneCommand stops using it */ static class StreamDataSerializer extends DataSerializer {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
398 private final OutputStream out;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
399 private final LogFacility log;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
400 private byte[] buffer;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
401
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
402 public StreamDataSerializer(LogFacility logFacility, OutputStream os) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
403 assert os != null;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
404 out = os;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
405 log = logFacility;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
406 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
407
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
408 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
409 public void write(byte[] data, int offset, int length) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
410 out.write(data, offset, length);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
411 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
412
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
413 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
414 public void writeInt(int... values) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
415 ensureBufferSize(4*values.length); // sizeof(int)
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
416 int idx = 0;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
417 for (int v : values) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
418 DataSerializer.bigEndian(v, buffer, idx);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
419 idx += 4;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
420 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
421 out.write(buffer, 0, idx);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
422 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
423
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
424 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
425 public void writeByte(byte... values) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
426 if (values.length == 1) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
427 out.write(values[0]);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
428 } else {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
429 out.write(values, 0, values.length);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
430 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
431 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
432
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
433 private void ensureBufferSize(int bytesNeeded) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
434 if (buffer == null || buffer.length < bytesNeeded) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
435 buffer = new byte[bytesNeeded];
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
436 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
437 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
438
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
439 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
440 public void done() {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
441 try {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
442 out.flush();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
443 out.close();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
444 } catch (IOException ex) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
445 log.dump(getClass(), Error, ex, "Failure to close stream");
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
446 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
447 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents: 494
diff changeset
448 }
10
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants
Artem Tikhomirov <tikhomirov.artem@gmail.com>
parents:
diff changeset
449 }