comparison design.txt @ 26:71a9ba42cee8

Memory-mapped files for bigger files. Defect reading number of bytes greater than size of the buffer fixed
author Artem Tikhomirov <tikhomirov.artem@gmail.com>
date Sun, 09 Jan 2011 15:59:54 +0100
parents da8ccbfae64d
children 565ce0835674
comparison
equal deleted inserted replaced
25:da8ccbfae64d 26:71a9ba42cee8
28 *input stream impl + lifecycle. Step forward with FileChannel and ByteBuffer, although questionable accomplishment (looks bit complicated, cumbersome) 28 *input stream impl + lifecycle. Step forward with FileChannel and ByteBuffer, although questionable accomplishment (looks bit complicated, cumbersome)
29 + dirstate.mtime 29 + dirstate.mtime
30 +calculate sha1 digest for file to see I can deal with nodeid 30 +calculate sha1 digest for file to see I can deal with nodeid
31 *.hgignored processing 31 *.hgignored processing
32 +Nodeid to keep 20 bytes always, Revlog.Inspector to get nodeid array of meaningful data exact size (nor heading 00 bytes, nor 12 extra bytes from the spec) 32 +Nodeid to keep 20 bytes always, Revlog.Inspector to get nodeid array of meaningful data exact size (nor heading 00 bytes, nor 12 extra bytes from the spec)
33 +DataAccess - implement memory mapped files,
33 34
34 DataAccess - collect debug info (buffer misses, file size/total read operations) to find out better strategy to buffer size detection. 35 DataAccess - collect debug info (buffer misses, file size/total read operations) to find out better strategy to buffer size detection. Compare performance.
35 DataAccess - implement memory mapped files, compare performance.
36 delta merge 36 delta merge
37 Changeset to get index (local revision number) 37 Changeset to get index (local revision number)
38 RevisionWalker (on manifest) and WorkingCopyWalker (io.File) talking to ? and/or dirstate 38 RevisionWalker (on manifest) and WorkingCopyWalker (io.File) talking to ? and/or dirstate
39
40
41
42 39
43 Status operation from GUI - guess, usually on a file/subfolder, hence API should allow for starting path (unlike cmdline, seems useless to implement include/exclide patterns - GUI users hardly enter them, ever) 40 Status operation from GUI - guess, usually on a file/subfolder, hence API should allow for starting path (unlike cmdline, seems useless to implement include/exclide patterns - GUI users hardly enter them, ever)
44 41
45 42
46 ??? encodings of fncache, .hgignore, dirstate 43 ??? encodings of fncache, .hgignore, dirstate
57 File access: 54 File access:
58 * NIO and mapped files - should be fast. Although seems to give less control on mem usage. 55 * NIO and mapped files - should be fast. Although seems to give less control on mem usage.
59 * Regular InputStreams and chunked stream on top - allocate List<byte[]>, each (but last) chunk of fixed size (depending on initial file size) 56 * Regular InputStreams and chunked stream on top - allocate List<byte[]>, each (but last) chunk of fixed size (depending on initial file size)
60 57
61 <<<<< 58 <<<<<
59
60 Tests:
61 DataAccess - readBytes(length > memBufferSize, length*2 > memBufferSize) - to check impl is capable to read huge chunks of data, regardless of own buffer size