hg4j: src/org/tmatesoft/hg/internal/DataAccessProvider.java annotate

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 598:d29d9dc6c128

Utilize the fact nodeids are very different and are read anyway to speed up reverse lookup

author	Artem Tikhomirov <tikhomirov.artem@gmail.com>
date	Fri, 03 May 2013 15:19:18 +0200
parents	ed243b668502
children	5daa42067e7c

rev	line source
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	1 /*
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	2 * Copyright (c) 2010-2013 TMate Software Ltd
74 6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	3 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	4 * This program is free software; you can redistribute it and/or modify
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	5 * it under the terms of the GNU General Public License as published by
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	6 * the Free Software Foundation; version 2 of the License.
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	7 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	8 * This program is distributed in the hope that it will be useful,
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	11 * GNU General Public License for more details.
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	12 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	13 * For information on how to redistribute this software under
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	14 * the terms of a license other than GNU General Public License
102 a3a2e5deb320 Updated contact address to support@hg4j.com Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 74 diff changeset	15 * contact TMate Software at support@hg4j.com
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	16 */
74 6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	17 package org.tmatesoft.hg.internal;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	18
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	19 import static org.tmatesoft.hg.util.LogFacility.Severity.Error;
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	20 import static org.tmatesoft.hg.util.LogFacility.Severity.Warn;
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	21
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	22 import java.io.File;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	23 import java.io.FileInputStream;
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	24 import java.io.FileNotFoundException;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	25 import java.io.FileOutputStream;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	26 import java.io.IOException;
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	27 import java.io.OutputStream;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	28 import java.nio.ByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	29 import java.nio.MappedByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	30 import java.nio.channels.FileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	31
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	32 import org.tmatesoft.hg.core.SessionContext;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	33 import org.tmatesoft.hg.util.LogFacility;
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	34
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	35 /**
74 6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	36 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	37 * @author Artem Tikhomirov
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	38 * @author TMate Software Ltd.
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	39 */
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	40 public class DataAccessProvider {
338 3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	41 /**
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	42 * Boundary to start using file memory mapping instead of regular file access, in bytes.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	43 * Set to 0 to indicate mapping files into memory shall not be used.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	44 * If set to -1, file of any size would be mapped in memory.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	45 */
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	46 public static final String CFG_PROPERTY_MAPIO_LIMIT = "hg4j.dap.mapio_limit";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	47 public static final String CFG_PROPERTY_MAPIO_BUFFER_SIZE = "hg4j.dap.mapio_buffer";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	48 public static final String CFG_PROPERTY_FILE_BUFFER_SIZE = "hg4j.dap.file_buffer";
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	49
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	50 private static final int DEFAULT_MAPIO_LIMIT = 100 * 1024; // 100 kB
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	51 private static final int DEFAULT_FILE_BUFFER = 8 * 1024; // 8 kB
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	52 private static final int DEFAULT_MAPIO_BUFFER = DEFAULT_MAPIO_LIMIT; // same as default boundary
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	53
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	54 private final int mapioMagicBoundary;
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	55 private final int bufferSize, mapioBufSize;
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	56 private final SessionContext context;
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	57 // not the right place for the property, but DAP is the only place currently available to RevlogStream to get the value
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	58 private final boolean shallMergePatches;
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	59
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	60 public DataAccessProvider(SessionContext ctx) {
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	61 context = ctx;
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	62 PropertyMarshal pm = new PropertyMarshal(ctx);
494 2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	63 mapioMagicBoundary = mapioBoundaryValue(pm.getInt(CFG_PROPERTY_MAPIO_LIMIT, DEFAULT_MAPIO_LIMIT));
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	64 bufferSize = pm.getInt(CFG_PROPERTY_FILE_BUFFER_SIZE, DEFAULT_FILE_BUFFER);
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	65 mapioBufSize = pm.getInt(CFG_PROPERTY_MAPIO_BUFFER_SIZE, DEFAULT_MAPIO_BUFFER);
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	66 shallMergePatches = pm.getBoolean(Internals.CFG_PROPERTY_PATCH_MERGE, false);
338 3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	67 }
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	68
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	69 public DataAccessProvider(SessionContext ctx, int mapioBoundary, int regularBufferSize, int mapioBufferSize) {
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	70 context = ctx;
494 2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	71 mapioMagicBoundary = mapioBoundaryValue(mapioBoundary);
27 b0a15cefdfd6 Cons args instead of fixed consts Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 26 diff changeset	72 bufferSize = regularBufferSize;
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	73 mapioBufSize = mapioBufferSize;
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	74 shallMergePatches = new PropertyMarshal(ctx).getBoolean(Internals.CFG_PROPERTY_PATCH_MERGE, false);
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	75 }
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	76
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	77 // TODO [post-1.1] find a better place for this option, it's unrelated to the DAP
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	78 public boolean shallMergePatches() {
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	79 return shallMergePatches;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	80 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	81
494 2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	82 // ensure contract of CFG_PROPERTY_MAPIO_LIMIT, for mapioBoundary == 0 use MAX_VALUE so that no file is memmap-ed
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	83 private static int mapioBoundaryValue(int mapioBoundary) {
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	84 return mapioBoundary == 0 ? Integer.MAX_VALUE : mapioBoundary;
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	85 }
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	86
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	87 public DataAccess createReader(File f) {
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	88 if (!f.exists()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	89 return new DataAccess();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	90 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	91 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	92 FileChannel fc = new FileInputStream(f).getChannel();
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	93 long flen = fc.size();
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	94 if (flen > mapioMagicBoundary) {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	95 // TESTS: bufLen of 1024 was used to test MemMapFileAccess
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	96 return new MemoryMapFileAccess(fc, flen, mapioBufSize, context.getLog());
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	97 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	98 // XXX once implementation is more or less stable,
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	99 // may want to try ByteBuffer.allocateDirect() to see
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	100 // if there's any performance gain.
338 3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	101 boolean useDirectBuffer = false; // XXX might be another config option
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	102 // TESTS: bufferSize of 100 was used to check buffer underflow states when readBytes reads chunks bigger than bufSize
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	103 return new FileAccess(fc, flen, bufferSize, useDirectBuffer, context.getLog());
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	104 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	105 } catch (IOException ex) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	106 // unlikely to happen, we've made sure file exists.
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	107 context.getLog().dump(getClass(), Error, ex, null);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	108 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	109 return new DataAccess(); // non-null, empty.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	110 }
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	111
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	112 public DataSerializer createWriter(File f, boolean createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	113 if (!f.exists() && !createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	114 return new DataSerializer();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	115 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	116 try {
538 dd4f6311af52 Commit: first working version Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 534 diff changeset	117 return new StreamDataSerializer(context.getLog(), new FileOutputStream(f, true));
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	118 } catch (final FileNotFoundException ex) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	119 context.getLog().dump(getClass(), Error, ex, null);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	120 return new DataSerializer() {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	121 public void write(byte[] data, int offset, int length) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	122 throw ex;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	123 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	124 };
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	125 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	126 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	127
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	128 private static class MemoryMapFileAccess extends DataAccess {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	129 private FileChannel fileChannel;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	130 private long position = 0; // always points to buffer's absolute position in the file
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	131 private MappedByteBuffer buffer;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	132 private final long size;
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	133 private final int memBufferSize;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	134 private final LogFacility logFacility;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	135
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	136 public MemoryMapFileAccess(FileChannel fc, long channelSize, int bufferSize, LogFacility log) {
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	137 fileChannel = fc;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	138 size = channelSize;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	139 logFacility = log;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	140 memBufferSize = bufferSize > channelSize ? (int) channelSize : bufferSize; // no reason to waste memory more than there's data
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	141 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	142
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	143 @Override
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	144 public boolean isEmpty() {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	145 return position + (buffer == null ? 0 : buffer.position()) >= size;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	146 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	147
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	148 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	149 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	150 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	151 return this;
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	152 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	153
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	154 @Override
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	155 public int length() {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	156 return Internals.ltoi(longLength());
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	157 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	158
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	159 @Override
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	160 public long longLength() {
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	161 return size;
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	162 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	163
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	164 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	165 public void longSeek(long offset) {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	166 assert offset >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	167 // offset may not necessarily be further than current position in the file (e.g. rewind)
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	168 if (buffer != null && /offset is within buffer/ offset >= position && (offset - position) < buffer.limit()) {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	169 // cast is ok according to check above
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	170 buffer.position(Internals.ltoi(offset - position));
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	171 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	172 position = offset;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	173 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	174 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	175 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	176
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	177 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	178 public void seek(int offset) {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	179 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	180 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	181
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	182 @Override
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	183 public void skip(int bytes) throws IOException {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	184 assert bytes >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	185 if (buffer == null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	186 position += bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	187 return;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	188 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	189 if (buffer.remaining() > bytes) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	190 buffer.position(buffer.position() + bytes);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	191 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	192 position += buffer.position() + bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	193 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	194 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	195 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	196
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	197 private void fill() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	198 if (buffer != null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	199 position += buffer.position();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	200 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	201 long left = size - position;
440 299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	202 for (int i = 0; i < 3; i++) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	203 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	204 buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, position, left < memBufferSize ? left : memBufferSize);
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	205 return;
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	206 } catch (IOException ex) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	207 if (i == 2) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	208 throw ex;
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	209 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	210 if (i == 0) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	211 // if first attempt failed, try to free some virtual memory, see Issue 30 for details
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	212 logFacility.dump(getClass(), Warn, ex, "Memory-map failed, gonna try gc() to free virtual memory");
440 299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	213 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	214 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	215 buffer = null;
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	216 System.gc();
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	217 Thread.sleep((1+i) * 1000);
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	218 } catch (Throwable t) {
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	219 logFacility.dump(getClass(), Error, t, "Bad luck");
440 299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	220 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	221 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	222 }
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	223 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	224
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	225 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	226 public void readBytes(byte[] buf, int offset, int length) throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	227 if (buffer == null \|\| !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	228 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	229 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	230 // XXX in fact, we may try to create a MappedByteBuffer of exactly length size here, and read right away
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	231 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	232 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	233 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	234 throw new IOException();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	235 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	236 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	237 buffer.get(buf, offset, length);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	238 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	239 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	240 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	241 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	242 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	243 length -= tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	244 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	245 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	246
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	247 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	248 public byte readByte() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	249 if (buffer == null \|\| !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	250 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	251 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	252 if (buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	253 return buffer.get();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	254 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	255 throw new IOException();
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	256 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	257
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	258 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	259 public void done() {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	260 buffer = null;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	261 if (fileChannel != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	262 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	263 fileChannel.close();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	264 } catch (IOException ex) {
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	265 logFacility.dump(getClass(), Warn, ex, null);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	266 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	267 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	268 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	269 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	270 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	271
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	272 // (almost) regular file access - FileChannel and buffers.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	273 private static class FileAccess extends DataAccess {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	274 private FileChannel fileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	275 private ByteBuffer buffer;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	276 private long bufferStartInFile = 0; // offset of this.buffer in the file.
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	277 private final long size;
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	278 private final LogFacility logFacility;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	279
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	280 public FileAccess(FileChannel fc, long channelSize, int bufferSizeHint, boolean useDirect, LogFacility log) {
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	281 fileChannel = fc;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	282 size = channelSize;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	283 logFacility = log;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	284 final int capacity = size < bufferSizeHint ? (int) size : bufferSizeHint;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	285 buffer = useDirect ? ByteBuffer.allocateDirect(capacity) : ByteBuffer.allocate(capacity);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	286 buffer.flip(); // or .limit(0) to indicate it's empty
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	287 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	288
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	289 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	290 public boolean isEmpty() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	291 return bufferStartInFile + buffer.position() >= size;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	292 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	293
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	294 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	295 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	296 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	297 return this;
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	298 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	299
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	300 @Override
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	301 public int length() {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	302 return Internals.ltoi(longLength());
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	303 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	304
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	305 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	306 public long longLength() {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	307 return size;
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	308 }
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	309
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	310 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	311 public void longSeek(long offset) throws IOException {
23 6f9aca1a97be Severe defect in buffer wrap on seek Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 10 diff changeset	312 if (offset > size) {
404 31a719b9f95e Provide more detailed information on erroneous file operation Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 338 diff changeset	313 throw new IllegalArgumentException(String.format("Can't seek to %d for the file of size %d (buffer start:%d)", offset, size, bufferStartInFile));
23 6f9aca1a97be Severe defect in buffer wrap on seek Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 10 diff changeset	314 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	315 if (offset < bufferStartInFile + buffer.limit() && offset >= bufferStartInFile) {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	316 // cast to int is safe, we've checked we fit into buffer
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	317 buffer.position(Internals.ltoi(offset - bufferStartInFile));
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	318 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	319 // out of current buffer, invalidate it (force re-read)
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	320 // XXX or ever re-read it right away?
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	321 bufferStartInFile = offset;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	322 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	323 buffer.limit(0); // or .flip() to indicate we switch to reading
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	324 fileChannel.position(offset);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	325 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	326 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	327
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	328 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	329 public void seek(int offset) throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	330 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	331 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	332
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	333 @Override
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	334 public void skip(int bytes) throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	335 final int newPos = buffer.position() + bytes;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	336 if (newPos >= 0 && newPos < buffer.limit()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	337 // no need to move file pointer, just rewind/seek buffer
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	338 buffer.position(newPos);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	339 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	340 //
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	341 longSeek(bufferStartInFile + newPos);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	342 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	343 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	344
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	345 private boolean fill() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	346 if (!buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	347 bufferStartInFile += buffer.limit();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	348 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	349 if (bufferStartInFile < size) { // just in case there'd be any exception on EOF, not -1
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	350 fileChannel.read(buffer);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	351 // may return -1 when EOF, but empty will reflect this, hence no explicit support here
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	352 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	353 buffer.flip();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	354 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	355 return buffer.hasRemaining();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	356 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	357
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	358 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	359 public void readBytes(byte[] buf, int offset, int length) throws IOException {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	360 if (!buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	361 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	362 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	363 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	364 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	365 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	366 throw new IOException(); // shall not happen provided stream contains expected data and no attempts to read past isEmpty() == true are made.
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	367 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	368 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	369 buffer.get(buf, offset, length);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	370 } else {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	371 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	372 fill();
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	373 }
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	374 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	375 length -= tail;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	376 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	377 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	378
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	379 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	380 public byte readByte() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	381 if (buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	382 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	383 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	384 if (fill()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	385 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	386 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	387 throw new IOException();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	388 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	389
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	390 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	391 public void done() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	392 if (buffer != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	393 buffer = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	394 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	395 if (fileChannel != null) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	396 try {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	397 fileChannel.close();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	398 } catch (IOException ex) {
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	399 logFacility.dump(getClass(), Warn, ex, null);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	400 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	401 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	402 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	403 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	404 }
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	405
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	406 public/XXX, private, once HgCloneCommand stops using it / static class StreamDataSerializer extends DataSerializer {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	407 private final OutputStream out;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	408 private final LogFacility log;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	409 private byte[] buffer;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	410
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	411 public StreamDataSerializer(LogFacility logFacility, OutputStream os) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	412 assert os != null;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	413 out = os;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	414 log = logFacility;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	415 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	416
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	417 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	418 public void write(byte[] data, int offset, int length) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	419 out.write(data, offset, length);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	420 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	421
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	422 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	423 public void writeInt(int... values) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	424 ensureBufferSize(4*values.length); // sizeof(int)
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	425 int idx = 0;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	426 for (int v : values) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	427 DataSerializer.bigEndian(v, buffer, idx);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	428 idx += 4;
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	429 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	430 out.write(buffer, 0, idx);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	431 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	432
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	433 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	434 public void writeByte(byte... values) throws IOException {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	435 if (values.length == 1) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	436 out.write(values[0]);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	437 } else {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	438 out.write(values, 0, values.length);
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	439 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	440 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	441
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	442 private void ensureBufferSize(int bytesNeeded) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	443 if (buffer == null \|\| buffer.length < bytesNeeded) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	444 buffer = new byte[bytesNeeded];
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	445 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	446 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	447
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	448 @Override
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	449 public void done() {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	450 try {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	451 out.flush();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	452 out.close();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	453 } catch (IOException ex) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	454 log.dump(getClass(), Error, ex, "Failure to close stream");
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	455 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	456 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	457 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	458 }

Mercurial > hg4j

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 598:d29d9dc6c128