jhg: src/org/tmatesoft/hg/internal/DataAccessProvider.java annotate

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 695:053bb4397bf9

Refactoring: nice Revlog.indexWalk() implementation

author	Artem Tikhomirov <tikhomirov.artem@gmail.com>
date	Mon, 05 Aug 2013 18:45:16 +0200
parents	12a4f60ea972
children

rev	line source
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	1 /*
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	2 * Copyright (c) 2010-2013 TMate Software Ltd
74 6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	3 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	4 * This program is free software; you can redistribute it and/or modify
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	5 * it under the terms of the GNU General Public License as published by
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	6 * the Free Software Foundation; version 2 of the License.
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	7 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	8 * This program is distributed in the hope that it will be useful,
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	9 * but WITHOUT ANY WARRANTY; without even the implied warranty of
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	10 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	11 * GNU General Public License for more details.
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	12 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	13 * For information on how to redistribute this software under
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	14 * the terms of a license other than GNU General Public License
102 a3a2e5deb320 Updated contact address to support@hg4j.com Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 74 diff changeset	15 * contact TMate Software at support@hg4j.com
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	16 */
74 6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	17 package org.tmatesoft.hg.internal;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	18
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	19 import static org.tmatesoft.hg.util.LogFacility.Severity.Error;
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	20 import static org.tmatesoft.hg.util.LogFacility.Severity.Warn;
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	21
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	22 import java.io.File;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	23 import java.io.FileInputStream;
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	24 import java.io.FileOutputStream;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	25 import java.io.IOException;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	26 import java.nio.ByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	27 import java.nio.MappedByteBuffer;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	28 import java.nio.channels.FileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	29
617 65c01508f002 Rollback support for commands that modify repository. Strategy to keep complete copy of a file being changed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 608 diff changeset	30 import org.tmatesoft.hg.core.HgIOException;
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	31 import org.tmatesoft.hg.core.SessionContext;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	32 import org.tmatesoft.hg.util.LogFacility;
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	33
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	34 /**
74 6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	35 *
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	36 * @author Artem Tikhomirov
6f1b88693d48 Complete refactoring to org.tmatesoft Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	37 * @author TMate Software Ltd.
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	38 */
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	39 public class DataAccessProvider {
338 3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	40 /**
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	41 * Boundary to start using file memory mapping instead of regular file access, in bytes.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	42 * Set to 0 to indicate mapping files into memory shall not be used.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	43 * If set to -1, file of any size would be mapped in memory.
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	44 */
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	45 public static final String CFG_PROPERTY_MAPIO_LIMIT = "hg4j.dap.mapio_limit";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	46 public static final String CFG_PROPERTY_MAPIO_BUFFER_SIZE = "hg4j.dap.mapio_buffer";
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	47 public static final String CFG_PROPERTY_FILE_BUFFER_SIZE = "hg4j.dap.file_buffer";
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	48
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	49 private static final int DEFAULT_MAPIO_LIMIT = 100 * 1024; // 100 kB
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	50 private static final int DEFAULT_FILE_BUFFER = 8 * 1024; // 8 kB
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	51 private static final int DEFAULT_MAPIO_BUFFER = DEFAULT_MAPIO_LIMIT; // same as default boundary
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	52
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	53 private final int mapioMagicBoundary;
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	54 private final int bufferSize, mapioBufSize;
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	55 private final SessionContext context;
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	56
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	57 public DataAccessProvider(SessionContext ctx) {
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	58 context = ctx;
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	59 PropertyMarshal pm = new PropertyMarshal(ctx);
494 2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	60 mapioMagicBoundary = mapioBoundaryValue(pm.getInt(CFG_PROPERTY_MAPIO_LIMIT, DEFAULT_MAPIO_LIMIT));
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	61 bufferSize = pm.getInt(CFG_PROPERTY_FILE_BUFFER_SIZE, DEFAULT_FILE_BUFFER);
909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	62 mapioBufSize = pm.getInt(CFG_PROPERTY_MAPIO_BUFFER_SIZE, DEFAULT_MAPIO_BUFFER);
338 3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	63 }
3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	64
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	65 public DataAccessProvider(SessionContext ctx, int mapioBoundary, int regularBufferSize, int mapioBufferSize) {
295 981f9f50bb6c Issue 11: Error log facility. SessionContext to share common facilities Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 158 diff changeset	66 context = ctx;
494 2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	67 mapioMagicBoundary = mapioBoundaryValue(mapioBoundary);
27 b0a15cefdfd6 Cons args instead of fixed consts Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 26 diff changeset	68 bufferSize = regularBufferSize;
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	69 mapioBufSize = mapioBufferSize;
584 ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	70 }
ed243b668502 Conditionally enable effective patch merge alternative for revlog reading Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 538 diff changeset	71
494 2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	72 // ensure contract of CFG_PROPERTY_MAPIO_LIMIT, for mapioBoundary == 0 use MAX_VALUE so that no file is memmap-ed
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	73 private static int mapioBoundaryValue(int mapioBoundary) {
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	74 return mapioBoundary == 0 ? Integer.MAX_VALUE : mapioBoundary;
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	75 }
2743641f2f12 Defect: use of 0 as configuration value for mapio boundary results in every file being memmap-ed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 456 diff changeset	76
606 5daa42067e7c Avoid mmap files when only few bytes are to be read Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 584 diff changeset	77 public DataAccess createReader(File f, boolean shortRead) {
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	78 if (!f.exists()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	79 return new DataAccess();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	80 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	81 try {
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	82 FileInputStream fis = new FileInputStream(f);
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	83 long flen = f.length();
606 5daa42067e7c Avoid mmap files when only few bytes are to be read Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 584 diff changeset	84 if (!shortRead && flen > mapioMagicBoundary) {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	85 // TESTS: bufLen of 1024 was used to test MemMapFileAccess
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	86 return new MemoryMapFileAccess(fis, flen, mapioBufSize, context.getLog());
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	87 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	88 // XXX once implementation is more or less stable,
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	89 // may want to try ByteBuffer.allocateDirect() to see
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	90 // if there's any performance gain.
338 3cfa4d908fc9 Add options to control DataAccessProvider, allow to turn off use of file memory mapping in particular to solve potential sharing violation (os file handle gets released on MappedByteByffer being GC'd, not on FileChannel.close()) Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 295 diff changeset	91 boolean useDirectBuffer = false; // XXX might be another config option
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	92 // TESTS: bufferSize of 100 was used to check buffer underflow states when readBytes reads chunks bigger than bufSize
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	93 return new FileAccess(fis, flen, bufferSize, useDirectBuffer, context.getLog());
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	94 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	95 } catch (IOException ex) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	96 // unlikely to happen, we've made sure file exists.
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	97 context.getLog().dump(getClass(), Error, ex, null);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	98 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	99 return new DataAccess(); // non-null, empty.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	100 }
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	101
618 7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	102 public DataSerializer createWriter(final Transaction tr, File f, boolean createNewIfDoesntExist) {
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	103 if (!f.exists() && !createNewIfDoesntExist) {
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	104 return new DataSerializer();
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	105 }
618 7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	106 // TODO invert RevlogStreamWriter to send DataSource here instead of grabbing DataSerializer
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	107 // to control the moment transaction gets into play and whether it fails or not
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	108 return new TransactionAwareFileSerializer(tr, f);
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	109 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	110
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	111 private static class MemoryMapFileAccess extends DataAccess {
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	112 private FileInputStream fileStream;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	113 private FileChannel fileChannel;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	114 private long position = 0; // always points to buffer's absolute position in the file
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	115 private MappedByteBuffer buffer;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	116 private final long size;
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	117 private final int memBufferSize;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	118 private final LogFacility logFacility;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	119
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	120 public MemoryMapFileAccess(FileInputStream fis, long channelSize, int bufferSize, LogFacility log) {
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	121 fileStream = fis;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	122 fileChannel = fis.getChannel();
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	123 size = channelSize;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	124 logFacility = log;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	125 memBufferSize = bufferSize > channelSize ? (int) channelSize : bufferSize; // no reason to waste memory more than there's data
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	126 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	127
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	128 @Override
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	129 public boolean isEmpty() {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	130 return position + (buffer == null ? 0 : buffer.position()) >= size;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	131 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	132
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	133 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	134 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	135 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	136 return this;
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	137 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	138
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	139 @Override
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	140 public int length() {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	141 return Internals.ltoi(longLength());
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	142 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	143
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	144 @Override
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	145 public long longLength() {
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	146 return size;
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	147 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	148
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	149 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	150 public void longSeek(long offset) {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	151 assert offset >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	152 // offset may not necessarily be further than current position in the file (e.g. rewind)
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	153 if (buffer != null && /offset is within buffer/ offset >= position && (offset - position) < buffer.limit()) {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	154 // cast is ok according to check above
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	155 buffer.position(Internals.ltoi(offset - position));
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	156 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	157 position = offset;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	158 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	159 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	160 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	161
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	162 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	163 public void seek(int offset) {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	164 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	165 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	166
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	167 @Override
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	168 public void skip(int bytes) throws IOException {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	169 assert bytes >= 0;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	170 if (buffer == null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	171 position += bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	172 return;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	173 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	174 if (buffer.remaining() > bytes) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	175 buffer.position(buffer.position() + bytes);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	176 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	177 position += buffer.position() + bytes;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	178 buffer = null;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	179 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	180 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	181
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	182 private void fill() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	183 if (buffer != null) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	184 position += buffer.position();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	185 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	186 long left = size - position;
440 299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	187 for (int i = 0; i < 3; i++) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	188 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	189 buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, position, left < memBufferSize ? left : memBufferSize);
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	190 return;
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	191 } catch (IOException ex) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	192 if (i == 2) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	193 throw ex;
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	194 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	195 if (i == 0) {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	196 // if first attempt failed, try to free some virtual memory, see Issue 30 for details
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	197 logFacility.dump(getClass(), Warn, ex, "Memory-map failed, gonna try gc() to free virtual memory");
440 299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	198 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	199 try {
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	200 buffer = null;
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	201 System.gc();
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	202 Thread.sleep((1+i) * 1000);
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	203 } catch (Throwable t) {
456 909306e412e2 Refactor LogFacility and SessionContext, better API for both Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 440 diff changeset	204 logFacility.dump(getClass(), Error, t, "Bad luck");
440 299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	205 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	206 }
299870249a28 Issue 30: bogus IOException for mmap file on linux Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 425 diff changeset	207 }
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	208 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	209
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	210 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	211 public void readBytes(byte[] buf, int offset, int length) throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	212 if (buffer == null \|\| !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	213 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	214 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	215 // XXX in fact, we may try to create a MappedByteBuffer of exactly length size here, and read right away
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	216 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	217 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	218 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	219 throw new IOException();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	220 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	221 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	222 buffer.get(buf, offset, length);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	223 } else {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	224 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	225 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	226 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	227 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	228 length -= tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	229 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	230 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	231
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	232 @Override
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	233 public byte readByte() throws IOException {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	234 if (buffer == null \|\| !buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	235 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	236 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	237 if (buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	238 return buffer.get();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	239 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	240 throw new IOException();
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	241 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	242
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	243 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	244 public void done() {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	245 buffer = null;
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	246 if (fileStream != null) {
654 12a4f60ea972 1) Console push tool. 2) Pass class to blame into FileUtils Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 619 diff changeset	247 new FileUtils(logFacility, this).closeQuietly(fileStream);
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	248 fileStream = null;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	249 fileChannel = null; // channel is closed together with stream
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	250 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	251 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	252 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	253
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	254 // (almost) regular file access - FileChannel and buffers.
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	255 private static class FileAccess extends DataAccess {
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	256 private FileInputStream fileStream;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	257 private FileChannel fileChannel;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	258 private ByteBuffer buffer;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	259 private long bufferStartInFile = 0; // offset of this.buffer in the file.
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	260 private final long size;
48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	261 private final LogFacility logFacility;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	262
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	263 public FileAccess(FileInputStream fis, long channelSize, int bufferSizeHint, boolean useDirect, LogFacility log) {
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	264 fileStream = fis;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	265 fileChannel = fis.getChannel();
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	266 size = channelSize;
425 48f993aa2f41 FIXMEs: exceptions, javadoc Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 420 diff changeset	267 logFacility = log;
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	268 final int capacity = size < bufferSizeHint ? (int) size : bufferSizeHint;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	269 buffer = useDirect ? ByteBuffer.allocateDirect(capacity) : ByteBuffer.allocate(capacity);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	270 buffer.flip(); // or .limit(0) to indicate it's empty
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	271 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	272
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	273 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	274 public boolean isEmpty() {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	275 return bufferStartInFile + buffer.position() >= size;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	276 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	277
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	278 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	279 public DataAccess reset() throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	280 longSeek(0);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	281 return this;
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	282 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	283
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	284 @Override
158 b413b16d10a5 Integer offsets and file length explictly, rather than casts throughout code. Inflater may benefit from total length hint, but shall calculate it by its own if needed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 157 diff changeset	285 public int length() {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	286 return Internals.ltoi(longLength());
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	287 }
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	288
9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	289 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	290 public long longLength() {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	291 return size;
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	292 }
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	293
51 9429c7bd1920 Try DataAccess to reach revision data instead of plain byte arrays Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 27 diff changeset	294 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	295 public void longSeek(long offset) throws IOException {
23 6f9aca1a97be Severe defect in buffer wrap on seek Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 10 diff changeset	296 if (offset > size) {
404 31a719b9f95e Provide more detailed information on erroneous file operation Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 338 diff changeset	297 throw new IllegalArgumentException(String.format("Can't seek to %d for the file of size %d (buffer start:%d)", offset, size, bufferStartInFile));
23 6f9aca1a97be Severe defect in buffer wrap on seek Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 10 diff changeset	298 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	299 if (offset < bufferStartInFile + buffer.limit() && offset >= bufferStartInFile) {
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	300 // cast to int is safe, we've checked we fit into buffer
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	301 buffer.position(Internals.ltoi(offset - bufferStartInFile));
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	302 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	303 // out of current buffer, invalidate it (force re-read)
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	304 // XXX or ever re-read it right away?
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	305 bufferStartInFile = offset;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	306 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	307 buffer.limit(0); // or .flip() to indicate we switch to reading
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	308 fileChannel.position(offset);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	309 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	310 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	311
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	312 @Override
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	313 public void seek(int offset) throws IOException {
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	314 longSeek(offset);
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	315 }
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	316
6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	317 @Override
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	318 public void skip(int bytes) throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	319 final int newPos = buffer.position() + bytes;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	320 if (newPos >= 0 && newPos < buffer.limit()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	321 // no need to move file pointer, just rewind/seek buffer
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	322 buffer.position(newPos);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	323 } else {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	324 //
420 6c22bdc0bdfd Respect long offsets in revlogs Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 414 diff changeset	325 longSeek(bufferStartInFile + newPos);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	326 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	327 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	328
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	329 private boolean fill() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	330 if (!buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	331 bufferStartInFile += buffer.limit();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	332 buffer.clear();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	333 if (bufferStartInFile < size) { // just in case there'd be any exception on EOF, not -1
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	334 fileChannel.read(buffer);
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	335 // may return -1 when EOF, but empty will reflect this, hence no explicit support here
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	336 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	337 buffer.flip();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	338 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	339 return buffer.hasRemaining();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	340 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	341
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	342 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	343 public void readBytes(byte[] buf, int offset, int length) throws IOException {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	344 if (!buffer.hasRemaining()) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	345 fill();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	346 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	347 while (length > 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	348 int tail = buffer.remaining();
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	349 if (tail == 0) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	350 throw new IOException(); // shall not happen provided stream contains expected data and no attempts to read past isEmpty() == true are made.
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	351 }
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	352 if (tail >= length) {
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	353 buffer.get(buf, offset, length);
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	354 } else {
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	355 buffer.get(buf, offset, tail);
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	356 fill();
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	357 }
26 71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	358 offset += tail;
71a9ba42cee8 Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 23 diff changeset	359 length -= tail;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	360 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	361 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	362
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	363 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	364 public byte readByte() throws IOException {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	365 if (buffer.hasRemaining()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	366 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	367 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	368 if (fill()) {
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	369 return buffer.get();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	370 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	371 throw new IOException();
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	372 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	373
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	374 @Override
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	375 public void done() {
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	376 buffer = null;
868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	377 if (fileStream != null) {
654 12a4f60ea972 1) Console push tool. 2) Pass class to blame into FileUtils Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 619 diff changeset	378 new FileUtils(logFacility, this).closeQuietly(fileStream);
619 868b2ffdcd5c Close FIS, not FileChannel, to clear both references to FileDescriptor right away Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 618 diff changeset	379 fileStream = null;
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	380 fileChannel = null;
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	381 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	382 }
382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	383 }
618 7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	384
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	385 /**
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	386 * Appends serialized changes to the end of the file
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	387 */
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	388 private static class TransactionAwareFileSerializer extends DataSerializer {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	389
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	390 private final Transaction transaction;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	391 private final File file;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	392 private FileOutputStream fos;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	393 private File transactionFile;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	394 private boolean writeFailed = false;
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	395
618 7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	396 public TransactionAwareFileSerializer(Transaction tr, File f) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	397 transaction = tr;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	398 file = f;
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	399 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	400
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	401 @Override
618 7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	402 public void write(byte[] data, int offset, int length) throws HgIOException {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	403 try {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	404 if (fos == null) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	405 transactionFile = transaction.prepare(file);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	406 fos = new FileOutputStream(transactionFile, true);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	407 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	408 fos.write(data, offset, length);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	409 fos.flush();
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	410 } catch (IOException ex) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	411 writeFailed = true;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	412 transaction.failure(transactionFile, ex);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	413 throw new HgIOException("Write failure", ex, transactionFile);
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	414 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	415 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	416
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	417 @Override
618 7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	418 public void done() throws HgIOException {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	419 if (fos != null) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	420 assert transactionFile != null;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	421 try {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	422 fos.close();
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	423 if (!writeFailed) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	424 // XXX, Transaction#done() assumes there's no error , but perhaps it's easier to
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	425 // rely on #failure(), and call #done() always (or change #done() to #success()
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	426 transaction.done(transactionFile);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	427 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	428 fos = null;
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	429 } catch (IOException ex) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	430 if (!writeFailed) {
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	431 // do not eclipse original exception
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	432 transaction.failure(transactionFile, ex);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	433 }
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	434 throw new HgIOException("Write failure", ex, transactionFile);
7c0d2ce340b8 Refactor approach how content finds it way down to a commit revision Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 617 diff changeset	435 }
534 243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	436 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	437 }
243202f1bda5 Commit: refactor revision creation code from clone command to work separately, fit into existing library structure Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: 494 diff changeset	438 }
10 382cfe9463db Dirstate parsing. DataAccess refactored to allow reuse and control over constants Artem Tikhomirov <tikhomirov.artem@gmail.com> parents: diff changeset	439 }

Mercurial > jhg

annotate src/org/tmatesoft/hg/internal/DataAccessProvider.java @ 695:053bb4397bf9