{dZddlmZddlZddlZddlZddlmZmZddl m Z ddl m Z ddl mZmZmZmZmZmZmZmZmZmZmZddlmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'dd l(m)Z)d Z*d Z+Gd d e,Z-Gdde-Z.Gdde.Z/Gdde-Z0Gdde,Z1Gdde1Z2dZ3dZ4d%dZ5Gdde,Z6Gdde,Z7dZ8Gd d!e-Z9d"Z:Gd#d$e.Z;y)&z/Git object store interfaces and implementation.BytesION) tree_changes walk_trees) NotTreeError)GitFile) CommitShaFileTagTreeZERO_SHA hex_to_sha sha_to_hexhex_to_filename S_ISGITLINK object_class valid_hexsha)PackPackData PackInflaterPackFileDisappearedload_pack_index_file iter_sha1pack_objects_to_datawrite_pack_headerwrite_pack_index_v2write_pack_datawrite_pack_objectcompute_file_sha PackIndexerPackStreamCopier)ANNOTATED_TAG_SUFFIXinfopackceZdZdZdZdZdZdZdZe dZ dZ d Z d Z d Zdd ZddZ ddZddZd d d dd fdZdZddZ ddZdZeedfdZdZy )BaseObjectStorezObject store interface.c|jDcgc])\}}||vr |jts |tk(s|+c}}Scc}}wN)itemsendswithr"r )selfrefsrefshas 'lib/third_party/dulwich/object_store.pydetermine_wants_allz#BaseObjectStore.determine_wants_allLsN#jjl * c$LL!568O *   s.Act||S)zIterate over the objects for the specified shas. Args: shas: Iterable object with SHAs Returns: Object iterator )ObjectStoreIterator)r+shass r/ iter_shaszBaseObjectStore.iter_shasUs#4..c,t|jz=Check if a particular object is present by SHA1 and is loose.)NotImplementedErrorcontains_looser+r.s r/r9zBaseObjectStore.contains_loose^s!$"5"566r5c,t|j)>Check if a particular object is present by SHA1 and is packed.)r8contains_packedr:s r/r=zBaseObjectStore.contains_packedbs!$"6"677r5cJ|j|xs|j|S)Check if a particular object is present by SHA1. This method makes no distinction between loose and packed objects. )r=r9r:s r/ __contains__zBaseObjectStore.__contains__fs% ##C(DD,?,?,DDr5ct)zIterable of pack objects.r8r+s r/packszBaseObjectStore.packsms "!r5c,t|jzObtain the raw text for an object. Args: name: sha for the object. Returns: tuple with numeric type and object contents. )r8get_rawr+names r/rGzBaseObjectStore.get_rawrs"$,,//r5cZ|j|\}}tj|||S)zObtain an object by SHA1.r.)rGr from_raw_string)r+r.type_numuncomps r/ __getitem__zBaseObjectStore.__getitem__{s*<<,&&&xSAAr5c,t|j5Iterate over the SHAs that are present in this store.)r8__iter__rCs r/rSzBaseObjectStore.__iter__s!$--00r5c,t|j))Add a single object to this object store.)r8 add_objectr+objs r/rVzBaseObjectStore.add_objects!$//22r5Nc,t|j)Add a set of objects to this object store. Args: objects: Iterable over a list of (object, path) tuples )r8 add_objectsr+objectsprogresss r/r[zBaseObjectStore.add_objectss "$"2"233r5c|dk(ry|j\}}} t|||||j|S#t$r |wxYw)zAdd pack data to this object store. Args: num_items: Number of items to add pack_data: Iterator over pack data tuples rNcompression_level)add_packrpack_compression_level BaseException)r+count pack_datar^fcommitaborts r/ add_pack_datazBaseObjectStore.add_pack_datasd A: ==?65  "&"="=  8O   G  s =Ac #DKt|||||||D]}|jj|jjf|jj|jjf|jj |jj ffyw)aFind the differences between the contents of two trees Args: source: SHA1 of the source tree target: SHA1 of the target tree want_unchanged: Whether unchanged files should be reported include_trees: Whether to include trees change_type_same: Whether to report files changing type in the same entry. Returns: Iterator over tuples with (oldpath, newpath), (oldmode, newmode), (oldsha, newsha) )want_unchanged include_treeschange_type_samerename_detectorN)roldpathnewmoder.)r+sourcetargetrlrmrnrochanges r/rzBaseObjectStore.tree_changess*#   )'-+ F&**//2&**//20  sBB c#Kt||dD]7\}}|jtj|jr|s4|9yw)aWIterate the contents of a tree and all subtrees. Iteration is depth-first pre-order, as in e.g. os.walk. Args: tree_id: SHA1 of the tree. include_trees: If True, include tree objects in the iteration. Returns: Iterator over TreeEntry namedtuples for all the objects in a tree. N)rrsstatS_ISDIR)r+tree_idrmentry_s r/iter_tree_contentsz"BaseObjectStore.iter_tree_contentss@#4$7HE1 &t||EJJ/G 8s AA A c|jSr(parentsrhs r/zBaseObjectStore.6>>r5c Rt|||||||}t|jdS)a`Find the missing objects required for a set of revisions. Args: haves: Iterable over SHAs already in common. wants: Iterable over SHAs of objects to fetch. shallow: Set of shallow commit SHA1s to skip progress: Simple progress function that will be called with updated progress strings. get_tagged: Function that returns a dict of pointed-to sha -> tag sha for including tags. get_parents: Optional function for getting the parents of a commit. Returns: Iterator over (sha, path) pairs. ) get_parentsN)MissingObjectFinderiternext) r+haveswantsshallowr^ get_taggedrdepthfinders r/find_missing_objectsz$BaseObjectStore.find_missing_objectss60%      # FKK&&r5cg}t|}|r4||vr"|j||j|t|}|r4|S)zFind which revisions this store has in common using graphwalker. Args: graphwalker: A graphwalker object. Returns: List of SHAs that are in common )rappendack)r+ graphwalkerrr.s r/find_common_revisionsz%BaseObjectStore.find_common_revisionssK;d{ S!${#C   r5cL|j||||}|j|S)a-Iterate over the contents of a pack file. Args: have: List of SHA1s of objects that should not be sent want: List of SHA1s of objects that should be sent shallow: Set of shallow commit SHA1s to skip progress: Optional progress reporting method )rr4)r+havewantrr^missings r/generate_pack_contentsz&BaseObjectStore.generate_pack_contentss)++D$J~~g&&r5c<t|j||||S)apGenerate pack data objects for a set of wants/haves. Args: have: List of SHA1s of objects that should not be sent want: List of SHA1s of objects that should be sent shallow: Set of shallow commit SHA1s to skip ofs_delta: Whether OFS deltas can be included progress: Optional progress reporting method )rr)r+rrrr^ ofs_deltas r/generate_pack_dataz"BaseObjectStore.generate_pack_datas%$  ' 'dGX F  r5c||}t|j}|tur|j\}}||}|tur|S)aPeel all tags from a SHA. Args: sha: The object SHA to peel. Returns: The fully-peeled SHA1 of a tag object, after peeling all intermediate tags; if the original ref does not point to a tag, this will equal the original SHA1. )r type_namer object)r+r.rX obj_classs r/peel_shazBaseObjectStore.peel_sha.sJ3i / 3 ZZNIss)C3 r5c|jSr(rrs r/rzBaseObjectStore.Crr5ct}t}g}|j||r`|jd}||vr|j|n6||vr2|j|||vrC||} |j|| |r`||fS)aCollect all ancestors of heads up to (excluding) those in common. Args: heads: commits to start from common: commits to end at, or empty set to walk repository completely get_parents: Optional function for getting the parents of a commit. Returns: a tuple (A, B) where A - all commits reachable from heads but not present in common, B - common (shared) elements that are directly reachable from heads r)setextendpopadd) r+headscommonrrbasescommitsqueueecmts r/_collect_ancestorsz"BaseObjectStore._collect_ancestors>s&% U ! AF{ ! '! A<1g [-.r5cy)z,Close any files opened by this object store.NrCs r/closezBaseObjectStore.closeasr5r()FFFNF)NN)NNT)__name__ __module__ __qualname____doc__r0r4r9r=r@propertyrDrGrOrSrVr[rjrr}rrrrrrrrrr5r/r&r&Is! /78E""0B 134:"H*1!'F 'BF "&u1 ! F;r5r&ceZdZddZedZdZdZdZdZ dZ dZ d Z ed Z d Zd Zd ZdZdZdZdZdZdZdZddZy)PackBasedObjectStorec i|_||_yr() _pack_cacherc)r+rcs r/__init__zPackBasedObjectStore.__init__gs&<#r5cgSr(rrCs r/ alternateszPackBasedObjectStore.alternatesks r5cR|jD] } ||vry y#t$rYwxYw)zpCheck if a particular object is present by SHA1 and is packed. This does not check alternates. TF)rDr)r+r.r$s r/r=z$PackBasedObjectStore.contains_packedos? JJD $; '  s  &&cx|j|s|j|ry|jD]}||vsyy)r?TF)r=r9r)r+r. alternates r/r@z!PackBasedObjectStore.__contains__|s@    $(;(;C(@Ii)r5c|jj|}||ur#||j|<|r|jyyy)z/Add a newly appeared pack to the cache by path.N)rgetr)r+ base_namer$ prev_packs r/_add_cached_packz%PackBasedObjectStore._add_cached_packsG$$((3 D *.D  Y '! !r5c||j}i|_|r'|j\}}|j|r&yyr()rpopitemr)r+ pack_cacherIr$s r/_clear_cached_packsz(PackBasedObjectStore._clear_cached_packss8%% %--/LT4 JJLr5c6|jjSr()rvaluesrCs r/_iter_cached_packsz'PackBasedObjectStore._iter_cached_packss&&((r5c,t|jr()r8_update_pack_cacherCs r/rz'PackBasedObjectStore._update_pack_caches!$"9"9::r5c$|jyr()rrCs r/rzPackBasedObjectStore.closes   "r5cht|jt|jzSzList with pack objects.)listrrrCs r/rDzPackBasedObjectStore.packss+D++-.d6M6M6O1PPPr5c#DK|jD] }|D]}|yw)z=Iterate over the SHAs of all the objects in alternate stores.N)r)r+ralternate_objects r/_iter_alternate_objectsz,PackBasedObjectStore._iter_alternate_objectss%I$- &&%.)s c,t|jz+Iterate over the SHAs of all loose objects.)r8_iter_loose_objectsrCs r/rz(PackBasedObjectStore._iter_loose_objectss!$":":;;r5c,t|jr()r8_get_loose_objectr:s r/rz&PackBasedObjectStore._get_loose_objects!$"8"899r5c,t|jr()r8_remove_loose_objectr:s r/rz)PackBasedObjectStore._remove_loose_objects!$";";<z.PackBasedObjectStore.repack..sE2D3C;2Ds)rrrrrDrIr)update iterobjectsr[rrrrrr) r+ loose_objectsr.rXr]p old_packsrIr$ consolidateds r/repackzPackBasedObjectStore.repacks'  ++-C   d44S9 :.*78-3C;-8*.**5*QQVVXq[* 5#//+JD$ NNE$2B2B2DE E,''0  l'')40 C  % %cff -!#//+JD$   d #, !7|#95s  EEc#K|j|jD]} |D]}||jD]}||j D]}|y#t$rYOwxYww)rRN)rrrrr)r+r$r.s r/rSzPackBasedObjectStore.__iter__s~ !++-D CI . ++-CI.//1CI2 '  s'$A6 A'5A6' A30A62A33A6c(|j|duS)zoCheck if a particular object is present by SHA1 and is loose. This does not check alternates. N)rr:s r/r9z#PackBasedObjectStore.contains_looses %%c*$66r5c|tk(r t|t|dk(rt|}|}n!t|dk(r|}d}nt d||j D]} |j |cS| t|}|j|}||j|jfS|jD]} |j |cS|jD]} |j |cSt|#ttf$rYwxYw#t$rYgwxYw#t$rYRwxYw)zObtain the raw fulltext for an object. Args: name: sha for the object. Returns: tuple with numeric type and object contents. (NzInvalid object name )r KeyErrorrrAssertionErrorrrGrrrrM as_raw_stringrr)r+rIr.hexshar$retrs r/rGzPackBasedObjectStore.get_rawsT 8 4. t9?T"CF Y"_CF T!CD D++-D ||C((. >%F$$V, ?<<!2!2!44 4++-D ||C((. I  ((00) v'12       s6&D  D".D1 DD" D.-D.1 D=<D=Nc8|jt|d|iS)zAdd a set of objects to this object store. Args: objects: Iterable over (object, path) tuples, should support __len__. Returns: Pack object of the objects written. r^)rjrr\s r/r[z PackBasedObjectStore.add_objectss""t!!#7#@T8TTr5)r()rrrrrrr=r@rrrrrrDrrrrrrrrSr9rGr[rr5r/rrfs=  ");#QQ' <:=5 8 7'RUr5rceZdZdZdfd ZdZedZedZ dZ dZ dZ d Z d Zd Zd Zd ZdZdZdZdZdZdZedZxZS)DiskObjectStorez+Git-style object store that exists on disk.ctt| |||_tjj |jt |_d|_||_ ||_ y)zOpen an object store. Args: path: Path of the object store. loose_compression_level: zlib compression level for loose objects pack_compression_level: zlib compression level for pack objects )rcN) superrrrqosjoinPACKDIRpack_dir _alternatesloose_compression_levelrc)r+rqrrc __class__s r/rzDiskObjectStore.__init__-sW ot-#9 .   TYY8 '>$&<#r5cPd|jjd|jdS)N<(z)>)rrrqrCs r/__repr__zDiskObjectStore.__repr__>s!^^44dii@@r5cx t|jddj} t|jddj} t|jddj}||||S#t$rd}YkwxYw#t$r|}YRwxYw#t$r|}Y9wxYw)N)scores compressionrslooseCompressionpackCompression)intrdecoder)clsrqconfigdefault_compression_levelrrcs r/ from_configzDiskObjectStore.from_configAs +(+ :~6==?) %  @&) :':;BBD' #  ?%( :'89@@B& " 402HII +(* % +  @&? # @  ?%> " ?s4)B )B)B+ BB B('B(+ B98B9c|j |jSg|_|jD]&}|jjt|(|jSr()r_read_alternate_pathsrr)r+rqs r/rzDiskObjectStore.alternatesWsZ    '## #..0D    # #OD$9 :1r5c #PK ttjj|jtdd}|5|j D]}|jd}|jdr&tjj|rtj|]tjtjjtj|j| dddy#t $rYywxYw#1swYyxYww)Nrrb #) rrrqrINFODIRFileNotFoundError readlinesrstrip startswithisabsfsdecodefsencode)r+rglines r/rz%DiskObjectStore._read_alternate_paths`s  TYYFMA {{5)??4(77==&++d++++bggll2;;tyy3I4&PQQ&Q!    Qs?D&9D D&CD D& DD&DD&D#D&c tjtjj|jttjj|jtd}t |d5} t|d}|5|j|jddd|jtj|dzdddtjj|s*tjj|j|}|jjt|y#t $rYwxYw#1swYxYw#t$rYwxYw#1swYxYw)z+Add an alternate path to this object store.rwbrNr)rmkdirrqrrFileExistsErrorropenwritereadrr"r rrr)r+rqalternates_pathrgorig_fs r/add_alternate_pathz"DiskObjectStore.add_alternate_pathos  HHRWW\\$))W5 6'',,tyy'<H _d +q +ot4GGFKKM* GGBKK%- .,ww}}T"77<< 40D t45   V%  , +sZAE>E> E/ E> E#./E> E E #E, (E>/ E;8E>:E;;E>>Fc tj|j}t }|D]p}|j ds|jds'tjj|ddz}||vsQ|dtd }|j|rg}|D]d}||jvsttjj|j|}|j|||j|<ft |j|z D]+}|jj!|j -|S#t$r|j gcYSwxYw)z4Read and iterate over new pack files and cache them.pack-.packr.idxN)rlistdirrrrrrr*rqsplitextrrrrrrr) r+pack_dir_contents pack_filesrIidx_name pack_name new_packsrgr$s r/rz"DiskObjectStore._update_pack_cachesC  " 4== 9 U %Dw'DMM',B77++D1!4v=00 $_G } 5INN9-& A(((BGGLL:;  &&*  #  T%%&3A     # ) ) +4/!  JJLI sEE10E1c.t|j|Sr()rrqr:s r/_get_shafile_pathz!DiskObjectStore._get_shafile_pathstyy#..r5c#FKtj|jD]{}t|dk7rtjtjj |j|D]*}tj ||z}t |s'|,}yw)N)rr2rqrrr"r)r+baserestr.s r/rz#DiskObjectStore._iter_loose_objectsstJJtyy)D4yA~ 277<< 4#@Akk$+.#C( B*sBB!cn|j|} tj|S#t$rYywxYwr()r:r from_pathrr+r.rqs r/rz!DiskObjectStore._get_loose_objects:%%c* $$T* *   s ( 44cLtj|j|yr()rremover:r:s r/rz$DiskObjectStore._remove_loose_objects $((-.r5cP |jtjj|j=|j tj|jjtj|jjy#t $rYnwxYwr() rrrqbasename _basenamerrrCdataindex)r+r$s r/rzDiskObjectStore._remove_packso   !1!1$..!AB  $))..! $**//"    s4B B%$B%ctd|D}|jd}tjj |j d|zS)Nc3&K|] }|d ywrNrrr{s r/rz5DiskObjectStore._get_pack_basepath..s958asciir/)rrrrqrr)r+entriessuffixs r/_get_pack_basepathz"DiskObjectStore._get_pack_basepaths>999w'ww||DMM7V+;<        .) ,,. nj9!%      s+H&H H HH#c6ddl}|j|jd\}}tj|d5}t ||j }t||||}|j|j||||cdddS#1swYyxYw)awAdd a new thin pack to this object store. Thin packs are packs that contain deltas with parents that exist outside the pack. They should never be placed in the object store directly, and always indexed and completed as they are copied. Args: read_all: Read function that blocks until the number of requested bytes are read. read_some: Read function that returns at least one byte, but may not return the number of bytes requested. Returns: A Pack object pointing at the now-completed thin pack in the objects/pack directory. rN tmp_pack_)dirprefixzw+bresolve_ext_ref delta_iter) tempfilemkstemprqrfdopenr rGr!verifyrm) r+read_all read_somervfdrqrgrcrbs r/ add_thin_packzDiskObjectStore.add_thin_packs} ## +#FD YYr5 !Q!!T\\BG%h 1QF MMO++AtVWE " ! !s A BBcpt|5}|j}|j|}|dz}tjj |s0t |d5}t|||jdddddd|jD]}|jk(s|cSdz}tjdk(r tj|tj||t!|} |j#|| | S#1swYxYw#1swYxYw#t$rYXwxYw)zMove a specific file containing a pack into the pack directory. Note: The file should be on the same file system as the packs directory. Args: path: Path to the pack file. r1r%Nr0rV)rsorted_entriesrQrrqexistsrrget_stored_checksumrDrFr^r_rCrr`rr) r+rqrrOrE index_namergr$rjrls r/ move_in_packzDiskObjectStore.move_in_pack's d^q&&(G..w7H!F*J77>>*-Z.!'7A4I4I4KL/ JJD~~) ( <<7 "  +& $ $(^  h 3!/. ^ %  s6ADD:DD)D DD&) D54D5cddl}|jjd\tjdfd}fd}||fS)Add a new pack to this object store. Returns: Fileobject to write to, a commit function to call when the pack is finished and an abort function. rNr0)rprPr%cjtjjtjj dkDrj StjyNr)rYrfsyncrrqgetsizerrC)rgr|rqr+sr/rhz(DiskObjectStore.add_pack..commitSsT GGI HHRL GGIwwt$q(((.. $r5cPjtjyr()rrrC)rgrqsr/riz'DiskObjectStore.add_pack..abort]s GGI IIdOr5)rvrwrrrx)r+rvrhrirgr|rqs` @@@r/rbzDiskObjectStore.add_packGsN ## g#FD IIb$   &%r5c|j|j}tjj |} tj |tjj|ryt|d5}|j|j|jdddy#t $rYlwxYw#1swYyxYw)z^Add a single object to this object store. Args: obj: Object to add Nr%r`) r:rrrqdirnamer&r'rrr)as_legacy_objectr)r+rXrqrprgs r/rVzDiskObjectStore.add_objectcs %%cff-ggood#  HHSM 77>>$   T4 A GG$$t7S7S$T !    ! sB2=,C2 B>=B>C c0 tj|tjtjj |dtjtjj |t ||S#t$rY}wxYw)Nr#)rr&r'rqrr)rrqs r/initzDiskObjectStore.initvsh  HHTN dF+, dG,-4y    sB BB)rr)rrrrrr  classmethodrrrrr-rr:rrrrrQrmr}rrbrVr __classcell__rs@r/rr*s5="AJJ*   R6*://#= ENF0@ 8&r5rc|eZdZdZfdZdZdZdZdZe dZ dZ d Z d Z d Zdd Zd ZdZdZxZS)MemoryObjectStorez.Object store that keeps all objects in memory.cFtt| i|_d|_y)Nr)rrr_datarc)r+rs r/rzMemoryObjectStore.__init__s  /1 &(#r5cpt|dk(r|St|dk(r t|Std|)Nrrz Invalid sha )rr ValueErrorr:s r/ _to_hexshazMemoryObjectStore._to_hexshas5 s8r>J X^c? "67 7r5c<|j||jvSr7)rrr:s r/r9z MemoryObjectStore.contains_loosess#tzz11r5cy)r<Frr:s r/r=z!MemoryObjectStore.contains_packedsr5cHt|jjSrQ)rrkeysrCs r/rSzMemoryObjectStore.__iter__sDJJOO%&&r5cgSrrrCs r/rDzMemoryObjectStore.packss  r5cb||j|}|j|jfSrF)rrMr)r+rIrXs r/rGzMemoryObjectStore.get_raws/4??4()||S..000r5cZ|j|j|jSr()rrcopyrHs r/rOzMemoryObjectStore.__getitem__s#zz$//$/05577r5c<|j|j|=y)z3Delete an object from this store, for testing only.N)rrrHs r/ __delitem__zMemoryObjectStore.__delitem__s JJtt, -r5cR|j|j|j<y)rUN)rrrrWs r/rVzMemoryObjectStore.add_objects XXZ 366r5c:|D]\}}|j|y)rZN)rV)r+r]r^rXrqs r/r[zMemoryObjectStore.add_objectss !IC OOC !r5c6tfd}d}||fS)aAdd a new pack to this object store. Because this object store doesn't support packs, we extract and add the individual objects. Returns: Fileobject to write to and a commit function to call when the pack is finished. ctjtjj }j t j|jD]}j|yr() r from_filergetvaluer[rr for_pack_datarGrV)rrXrgr+s r/rhz*MemoryObjectStore.add_pack..commitsW""71::<#8!&&(CA GGI#11!T\\B$Cr5cyr(rrr5r/riz)MemoryObjectStore.add_pack..aborts r5r)r+rhrirgs` @r/rbzMemoryObjectStore.add_packs% I %  &%r5ct|}|jdt|t|t|j zt |d}|j D]5}t|dk(sJ|j |\}}t||||7|j}|j|y)zComplete a thin pack by adding external references. Args: f: Open file object for the pack. indexer: A PackIndexer for indexing the pack. rrSrTrrKN) rrWrrrXrrGrr\r)) r+rgrcrOrdrerMrGrhs r/rmz%MemoryObjectStore._complete_thin_packsw- q !S\C0@0@0B,CCD#1c2'')Gw<2% %%!\\'2NHd a4W =*>># r5c|j\}}} t||j}t||||}|j |j |||y#t $r |wxYw)aAdd a new thin pack to this object store. Thin packs are packs that contain deltas with parents that exist outside the pack. Because this object store doesn't support packs, we extract and add the individual objects. Args: read_all: Read function that blocks until the number of requested bytes are read. read_some: Read function that returns at least one byte, but may not return the number of bytes requested. rrrtN)rbr rGr!ryrmrd)r+rzr{rgrhrircrbs r/r}zMemoryObjectStore.add_thin_packst ==?65 !!T\\BG%h 1QF MMO  $ $Q 0 H   G  s AA&&A8r()rrrrrrr9r=rSrrDrGrOrrVr[rbrmr}rrs@r/rrs[8) 82'18.(! ,0r5rceZdZdZdZy)ObjectIteratorz%Interface for iterating over objects.c,t|jr()r8rrCs r/rzObjectIterator.iterobjects s!$"2"233r5N)rrrrrrr5r/rrs /4r5rcLeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z y ) r2z3ObjectIterator that works on top of an ObjectStore.c.||_||_g|_y)zCreate a new ObjectIterator. Args: store: Object store to retrieve from sha_iter: Iterator over (sha, path) tuples N)storesha_iter_shas)r+rrs r/rzObjectStoreIterator.__init__s    r5c#bK|jD]\}}|j||fyw)z&Yield tuple with next object and path.N)itershasrrAs r/rSzObjectStoreIterator.__iter__s-IC**S/4' ')s-/c#(K|D] \}}| yw)zIterate over just the objects.Nr)r+orqs r/rzObjectStoreIterator.iterobjects"sGAtGsc#K|jD]}||jD]!}|jj||#yw)zIterate over the SHAs.N)rrrr:s r/rzObjectStoreIterator.itershas's>::CI==C JJ  c "I!sAA c2|tk(ry||jvS)aCheck if an object is present. Note: This checks if the object is present in the underlying object store, not if it would be yielded by the iterator. Args: needle: SHA1 of the object to check for F)r r)r+needles r/r@z ObjectStoreIterator.__contains__/s X ##r5c |j|S)zFind an object by SHA1. Note: This retrieves the object from the underlying object store. It will also succeed if the object would not be returned by the iterator. )r)r+keys r/rOzObjectStoreIterator.__getitem__=szz#r5cFtt|jS)zReturn the number of objects.)rrrrCs r/__len__zObjectStoreIterator.__len__Fs4 ())r5cVddl}|jdt|jS)NrzUse bool() instead.)warningswarnDeprecationWarning_empty)r+rs r/emptyzObjectStoreIterator.emptyJs! +-?@{{}r5cZ|j} t|y#t$rYywxYw)NFT)rr StopIteration)r+its r/rzObjectStoreIterator._emptyPs2 ]]_  H  s  **c$|j S)z*Indicate whether this object has contents.)rrCs r/__bool__zObjectStoreIterator.__bool__Ys;;=  r5N)rrrrrrSrrr@rOrrrrrr5r/r2r2s7= (   $* !r5r2cl||}t|ts t||j||S)zLook up an object in a Git tree. Args: lookup_obj: Callback for retrieving object by SHA1 root_sha: SHA1 of the root tree path: Path to lookup Returns: A tuple of (mode, SHA) of the resulting path. ) isinstancer r lookup_path) lookup_objroot_sharqtrees r/tree_lookup_pathr^s6 h D dD !8$$   J --r5c||}|jD]K\}}}t|r||vs|j|tj|s?t |||My)zCollect SHA1s of files and directories for specified tree. Args: obj_store: Object store to get objects by SHA from tree_sha: tree reference to walk kset: set to fill with references to files and directories N) iteritemsrrrxry_collect_filetree_revs) obj_storetree_shaksetfiletreerIrsr.s r/rrmsX"H#--/dC4 S_ HHSM||D!&y#t< 0r5ct}t}t}|D]} ||}t|tr|j|+t|trC|j||j d}t ||g|\} } }|| z}|| z}||z}~|j||||fS#t$r|sYwxYw)a4Split object id list into three lists with commit, tag, and other SHAs. Commits referenced by tags are included into commits list as well. Only SHA1s known in this repository will get through, and unless ignore_unknown argument is True, KeyError is thrown for SHA1 missing in the repository Args: obj_store: Object store to get objects by SHA1 from lst: Collection of commit and tag SHAs ignore_unknown: True to skip SHA1 missing in the repository silently. Returns: A tuple of (commits, tags, others) SHA1s )ignore_unknown)rrr rr r_split_commits_and_tagsr) rlstrrtagsothersrrtaggedcts r/rr}seG 5D UF  ! A !V$ AAs# !1x1a1  !  1 '( T6 ""# !" sB77 CCc2eZdZdZddddfdZdZdZeZy)ra1Find the objects missing from another object store. Args: object_store: Object store containing at least all objects to be sent haves: SHA1s of commits not to send (already present in target) wants: SHA1s of commits to send progress: Optional function to report progress to. get_tagged: Function that returns a dict of pointed-to sha -> tag sha for including tags. get_parents: Optional function for getting the parents of a commit. tagged: dict of pointed-to sha -> tag sha for including tags Nc|jSr(rrs r/rzMissingObjectFinder.rr5c||_| t}||_t||d\}} } t||d\} } } |j |||jd}|j | |||j\}}t|_|D]C}|j j |||}t||j|j E| D]}|j j || j| }| j| }|j|}|j|}t|Dcgc]}|ddf c}|_ | d|_ n||_ |xr|xsi|_ ycc}w)NTF)rrrcyr(r)xs r/rz.MissingObjectFinder.__init__..sdr5) object_storer _get_parentsrrsha_donerrr differenceunionobjects_to_sendr^_tagged)r+rrrrr^rr have_commits have_tags have_others want_commits want_tags want_others all_ancestorsmissing_commitscommon_commitshrr missing_tagsmissing_othersws r/rzMissingObjectFinder.__init__s) ?eG' 0G %0 , i0G %0 , i %77 't7H7H8  +7*I*I  )) +J+ '   A MM  a q/C "<4== I A MM  a !++I6 $// < %%l3 N+"e#DeQe$4e#DE  *DM$DM!2jl8b $Es Fc|jj|Dcgc]}|d|jvs|c}ycc}wr)rrr)r+rOrs r/add_todozMissingObjectFinder.add_todos7 ##$U1qtt}}?TQ$UV$Us==c H |jsy|jj\}}}||jvrn;|s|j|}t |t r |j |jddfgnt |trX|j |jDcgc]*\}}}t|s||tj| f,c}}}n2t |tr"|j |jdddfg||jvr"|j |j|ddfg|jj!||j#dt%|jzj'd||fScc}}}w)NTFrzcounting objects: %d rN)rrrrrr rrr rrrxryr rrrr^rencode)r+r.rIleafrnmss r/rzMissingObjectFinder.nextsm'' $ 4 4 8 8 : S$$--'  !!#&A!V$ E234At$ (){{}'4GAq!*1~A4<<?23'4As#  T59:; $,,  MMDLL-tT:; < # /#dmm2DDLLWUVT{s-/F )rrrrrrr__next__rr5r/rrs. &1A9FW6Hr5rc(eZdZdZddZdZdZeZy)ObjectStoreGraphWalkerzGraph walker that finds what commits are missing from an object store. :ivar heads: Revisions without descendants in the local repo :ivar get_parents: Function to retrieve parents in the local repo Ncft||_||_i|_| t}||_y)zCreate a new instance. Args: local_heads: Heads to start search with get_parents: Function for finding the parents of a SHA1. N)rrrrr)r+ local_headsrrs r/rzObjectStoreGraphWalker.__init__ s2% & ?eG r5ct|dk7rtd|zt|g}|jr|D],}||jvs|jj |.t}|D]?}|j j |}||j|d|j |<A|sy|}|jryy)z@Ack that a revision and its ancestors are present in the source.rzunexpected sha %r receivedN)rrrrrCrrr)r+r. ancestorsa new_ancestorspss r/rzObjectStoreGraphWalker.ack.s s8r>9C?@ @J jj ?JJ%%a(  EM\\%%a(>!((,"& Q !%I#jjr5c|jrt|jj}|j|}||j|<|jj |Dcgc]}||jvs|c}|Sycc}w)z.Iterate over ancestors of heads in the target.N)rrrrr)r+rrrs r/rzObjectStoreGraphWalker.nextHsq ::**.."C!!#&B "DLL  JJ  "F"Q0Eq"F GJGs B4Br()rrrrrrrrrr5r/rrs &4Hr5rci}|D]@\}}} |jdd\}}|j|gj|||fB|j D]O\} } ||| d} t|| | } t| dk(r|| =1tj| jf|| <Q|j||S#t$r|||=n||f||<YwxYw#t $r t } YwxYw)aCommit a specified set of changes to a tree structure. This will apply a set of changes on top of an existing tree, storing new objects in object_store. changes are a list of tuples with (path, mode, object_sha). Paths can be both blobs and trees. See the mode and object sha to None deletes the path. This method works especially well if there are only a small number of changes to a big tree. For a large number of changes to a large tree, use e.g. commit_tree. Args: object_store: Object store to store new objects in and retrieve old ones from. tree: Original tree root changes: changes to apply Returns: New tree root object /rr) split setdefaultrrr)rr commit_tree_changesrrxS_IFDIRrrV) rrchangesnested_changesrqnew_moderdrsubpathrI subchanges orig_subtreesubtrees r/rrUs.N%,!x X!%D!!4 Wg  % %gr 2 9 97Hg:V W&-+002j "'T 1 6L&lL*M w<1 T ,, 3DJ3D! K% 1J&0T  1 "6L "s#B= C=CCC/.C/cNeZdZdZd dZdZd dZedZdZ dZ d Z d Z y) OverlayObjectStorez5Object store that can overlay multiple object stores.Nc ||_||_yr()r add_store)r+rr!s r/rzOverlayObjectStore.__init__s "r5cz|jt|j|jj|Sr()r!r8rV)r+rs r/rVzOverlayObjectStore.add_objects0 >> !%doo6 6~~((00r5c||jt|j|jj||Sr()r!r8rVr[r\s r/r[zOverlayObjectStore.add_objectss2 >> !%doo6 6~~))'8<z BucketBasedObjectStore._get_packs!$..11r5cVt|j}g}|D]B}||jvs|j|}|j |||j|<Dt|j|z D]+}|jj |j -|Sr()rr<rr>rrr)r+r5r8rgr$s r/rz)BucketBasedObjectStore._update_pack_caches..01  A(((~~a(  &&*  #  T%%&3A     # ) ) +4r5ctr(rB)r+rE pack_filerks r/ _upload_packz#BucketBasedObjectStore._upload_packs!!r5c\ddljfd}|jfS)rrNcjdk(rjyjdtj}|j }t d|Djd} j}|j}t||||jdt|dz|} jD]G}|j|jk(s%|j|j|cSjd|jd j||tj||} j!|||S)Nrc3&K|] }|d ywrKrrLs r/rzBBucketBasedObjectStore.add_pack..commit..s ?weqwrMrNr1)r[rrWrrIrrrSpooledTemporaryFilerrrrDrBr from_objectsr) rrOrEidxfchecksumidxr$rlpfr+rvs r/rhz/BucketBasedObjectStore.add_pack..commits5wwyA~  GGAJ"%A&&(G ?w ??FFwOH002D,,.H gx 8 IIaL&x&'8$?C ++-1F1F1HHGGIIIKK # GGAJ IIaL   hD 1**1c2J  ! !(J 7 r5)rvrFr)r+rhrKrvs` @@r/rbzBucketBasedObjectStore.add_packs.   * * , 46288##r5N) rrrrrrrrr<r>rrBrbrr5r/r6r6s4 592 "%$r5r6r)rSs .6 !     .  Y;fY;zAU?AUHT*Tn DDN4V4L!.L!^ . = &#Rp&pf9V9x-`22j O$1O$r5