7TdZddlmZddlmZddlmZddlZddlZddlZddlZddl m Z ddl m Z ddl m Z dd lmZdd lmZdd lmZd Zd ZdZdZdZdZdZdZdZdZdZdZdZ GddZ!ejDdddgZ#dZ$dZ%ejLd !d"Z'd#Z( d1d$Z)ejDd%gd&Z*d2d'Z+ d3d(Z,d)Z-d*Z.d+Z/d,Z0d2d-Z1d.Z2d/Z3d0Z4y)4z%POSIX utilities for storage commands.)absolute_import)division)unicode_literalsN)errors) storage_url)resource_reference)log)function_result_cache) platformsz;UID in {} metadata doesn't exist on current system. UID: {}z;GID in {} metadata doesn't exist on current system. GID: {}z|Insufficient access to local destination to apply {}. User {} owns file, but owner does not have read permission in mode {}.zInsufficient access to local destination to apply {}. Group {} would own file, but group does not have read permission in mode {}.zInsufficient access to local destination to apply {}. UID {} is not owner of file, and user is not in a group that owns the file. Users in "other" category do not have read permission in mode {}.zgoog-reserved-file-atimezgoog-reserved-posix-gidzgoog-reserved-posix-modezgoog-reserved-file-mtimezgoog-reserved-posix-uidiQcPt|dd}|dd}ddt|z z|zS)zCTakes base ten integer, converts to octal, and removes extra chars.N0)octlen) base_ten_int oct_stringpermission_bytess 4lib/googlecloudsdk/command_lib/storage/posix_util.py"convert_base_ten_to_base_eight_strr=s=< $*_ C()) *-= ==ct|dS)zHTakes string representing integer in octal and converts to base ten int.)int)base_eight_strs r&convert_base_eight_str_to_base_ten_intrHs ^Q rcBeZdZdZdZedZedZdZdZ y) PosixModez(Stores POSIX mode in all useful formats.c ||_||_y)z8Initializes class. Prefer the 'from' constructors below.N)rr)selfrrs r__init__zPosixMode.__init__Qs$D(DrcBt|}tt||S)z-Initializes class from base ten int. E.g. 73.)rrr)clsrrs rfrom_base_ten_intzPosixMode.from_base_ten_intVs'8 EN .~> PPrc,tt||S)z=Initializes class from base eight (octal) string. E.g. '111'.)rr)r$rs rfrom_base_eight_strzPosixMode.from_base_eight_str^s .~> PPrct|t|stS|j|jk(xr|j|jk(S)N) isinstancetypeNotImplementedrr)r!others r__eq__zPosixMode.__eq__dsH eT$Z (    !3!3 3 8   5#7#7 79rcNdj|j|jS)Nz&(base-ten int: {}, base-eight str: {}))formatrr)r!s r__repr__zPosixMode.__repr__js& 3 : : 4.. 00rN) __name__ __module__ __qualname____doc__r" classmethodr%r'r-r0rrrrNs@0) PPPP 9 0rrSystemPosixData default_mode user_groupscd}tjd}tj|||z }|dz}tj|S)zDGets default permissions files are created with as PosixMode object.ii)osumaskrr%)max_permissions current_umaskmodemode_without_executions r_get_default_moderBysL/((5/-((= = ($ %<  $ $%; <B )maxsizectjjr tddSt }t }t||S)z1Gets POSIX info that should only be fetched once.N)r OperatingSystem IsWindowsr7rBrQ)r8r9s rget_system_posix_datarWs@((* 4 &&"$, "+ { 33rc4|rtj||)z8Deletes file before raising error if file path provided.)r<remove)error delete_paths r"_raise_error_and_maybe_delete_filer\sIIk +rcD|xs t|\}}}}}||cxur|cxur ntjjryt j dk(ryddl}ddl} |jj} | | j|| |j#|| |j(} n|} |xst j*} ||t j*k(r^| j,t.j0zrytjt2j| | | j4} t!| ||||j6vrb| j,t.j8zrytjt:j| |dn|| j4} t!| || j,t.j<zrytjt>j| | | j4} t!| |y#t$r9tjtj| |} t!| |YwxYw#tt$f$r9tjt&j| |} t!| |YwxYw)a8Detects permissions causing inaccessibility. Can delete invalid file. Args: system_posix_data (SystemPosixData): Helps determine if file will be made inaccessible in local environment. resource (ObjectResource): Contains URL used for messages and custom POSIX metadata used to determine if setting invalid file permissions. delete_path (str|None): If present, will delete file before raising error. Useful if file has been downloaded and needs to be cleaned up. known_posix (PosixAttributes|None): Use pre-parsed POSIX data instead of extracting from source. Not super important here because the source is a cloud object and doesn't require an `os.stat` call to harvest metadata, but it would be strange if we used `known_posix` for callers and only `resource` here, especially if the values were different (which they shouldn't be). Be careful using this because, if the data is wrong, it could mess with these safety checks. Raises: SystemPermissionError: Has explanatory message about issue. Nrz[user primary group]) (get_posix_attributes_from_cloud_resourcer rUrVr<geteuidrDrEr url_stringrGKeyErrorrSystemPermissionError_MISSING_UID_FORMATr/r\getgrgid OverflowError_MISSING_GID_FORMATr8rFrstatS_IRUSR%_INSUFFICIENT_USER_READ_ACCESS_FORMATrr9S_IRGRP&_INSUFFICIENT_GROUP_READ_ACCESS_FORMATS_IROTH&_INSUFFICIENT_OTHER_READ_ACCESS_FORMAT)system_posix_dataresourcer[ known_posix_uidgidr@rDrEr`rZ mode_to_set uid_to_sets r!raise_if_invalid_file_permissionsrvs^:G=hG!QS$ S D Y%>%>%H%H%J ZZ\Q   ##..*_= ll3  _= ll3 \#00KK!biik*[C299;&$,,.  ( (-44  K$>$>  E 'uk:[C,888$,,.  ( (.55 &)k "s  & &  E'uk: ,  & &,33 j+"<"< % %UK8o =**  $ $Z 5e) < = m $=**  $ $Z 5e) < =s%<HI>IIAJJPosixAttributes)atimemtimerrrsr@c | xs tjtjv}tj||\ }}}}}}}}}}t||||tj |S)z3Takes file path and returns PosixAttributes object.follow_symlinks)r<rgsupports_follow_symlinksrwrr%) file_pathpreserve_symlinksr|r@rqrrrsrxrys rget_posix_attributes_from_filer(sr Irwwb.I.II131-$1ac1eUA sC"44T: <| xs tj(tjv} tj(|| || |j*|j*j,|j*j,k7rR| xs tj.tjv} tj.||j*j,| yyy)aSets custom POSIX attributes on file if the final metadata will be valid. This function is typically called after downloads. `raise_if_invalid_file_permissions` should have been called before initiating a download, but we call it again here to be safe. Args: system_posix_data (SystemPosixData): System-wide POSIX. Helps fill in missing data and determine validity of result. source_resource (resource_reference.ObjectResource): Source resource with POSIX attributes to apply. destination_resource (resource_reference.FileObjectResource): Destination resource to apply POSIX attributes to. known_source_posix (PosixAttributes|None): Use pre-parsed POSIX data instead of extracting from source. known_destination_posix (PosixAttributes|None): Use pre-parsed POSIX data instead of extracting from destination. preserve_symlinks (bool): Whether symlinks should be preserved rather than followed. Raises: SystemPermissionError: Custom metadata asked for file ownership change that user did not have permission to perform. Other permission errors from OS functions are possible. Also see `raise_if_invalid_file_permissions`. )rpNFr{rz(Root permissions required to set UID {}.)r resource_namervr^rrxryr<utimer}r rUrVrrr_rYrrbr/rschownr@rchmod)rnsource_resourcedestination_resourceknown_source_posixknown_destination_posixrdestination_pathcustom_posix_attributesexisting_posix_attributesrxneed_utime_callryr|rrneed_chown_callrss r%set_posix_attributes_on_file_if_validr4sB*55CC#$ C 1/ B M '(8:K L ""* % + +EO # ) )E%%)B)H)HH""* % + +E # ) )E L " ( (,E,K,K K  L1L1L!LHH u~O((*   ( # ' 'CO ! % %C##'@'D'DD '+++ 0Aii !  ( ( 4 ; ;C @   ( # ' 'C ! % %C H " & &*C*G*G G  L1L1L!LHH sCI!!-""// " ' ' 4 45 L1L1L!LHH$$11' 5.rc |jr|jj|y t|j|}|dkrHt j dj |jj||j|y|tjjtjjjtzkDrHt j dj ||jj|j|y|S#t$rJt j dj |jj||j|YywxYw)z1Finds, validates, and returns a POSIX time value.N6{} metadata did not contain a numeric value for {}: {}rz/Found negative time value in {} metadata {}: {}z^Found {} value in {} metadata that is more than one day in the future from the system time: {}) custom_fieldsgetr ValueErrorr warningr/rr`datetimenowtimezoneutc timestamp_SECONDS_PER_DAY)rokeyrs r"_extract_time_from_custom_metadatarsQ   8#9#9#=#=c#B#J H**3/0I]KK9@@  + +S(2H2H2M  h//334>>@ KK ++16 %%00(2H2H2M,   5 KK@GG  + +S(2H2H2M   sD((AE;:E;c |jr|jj|y t|j|}|dkrHt j dj |jj||j|y|S#t$rJt j dj |jj||j|YywxYw)z/Finds, validates, and returns a POSIX ID value.Nrrz-Found negative ID value in {} metadata {}: {}) rrrrr rr/rr`)rorposix_ids r _extract_id_from_custom_metadatars   8#9#9#=#=c#B#J 8))#./H\KK7>>  + +S(2H2H2M   / KK@GG  + +S(2H2H2M   sBAC$#C$c d|jr|jjty tj |jtS#t $rRt jdj|jjt|jtYywxYw)z1Finds, validates, and returns a POSIX mode value.NzG{} metadata did not contain a valid permissions octal string for {}: {}) rrMODE_METADATA_KEYrr'rr rr/rr`)ros r"_extract_mode_from_custom_metadatars     # #$5 6 >    ( (01  KK f  + +   " "#4 5  s%AAB/.B/ct|t}t|t}t|t}t|t }t |}t|||||S)akParses metadata_dict and returns PosixAttributes. Note: This parses an object's *custom* metadata with user-set fields, not the full metadata with provider-set fields. Args: resource (ObjectResource): Contains URL to include in logged warnings and custom metadata to parse. Returns: PosixAttributes object populated from metadata_dict. )rATIME_METADATA_KEYMTIME_METADATA_KEYrUID_METADATA_KEYGID_METADATA_KEYrrw)rorxryrrrsr@s rr^r^sU -X7I J% ,X7I J%(3CD#(3CD# +H 5$ sC 66rct|tjr t|St|tjr t |j j|Stjdj|j)z,Parses unknown resource type for POSIX data.zECan only retrieve POSIX attributes from file or cloud object, not: {}) r)rObjectResourcer^FileObjectResourcerrrrInvalidUrlErrorr/ TYPE_STRING)rors r"get_posix_attributes_from_resourcersx,;;< 3H ==,??@ )**,=   4 45 rc|jt|j|t<|jt|j|t<|j t|j |t <|jt|j|t<|j|jj|t<yy)z7Updates custom metadata_dict with PosixAttributes data.N) rxstrrryrrrrrsrr@rr) metadata_dictposix_attributess r1update_custom_metadata_dict_with_posix_attributesr's'(+,<,B,B(CM$%'(+,<,B,B(CM$%%&)*:*>*>&?M"#%&)*:*>*>&?M"#&'7'<'<'K'KM#$'rct|tjr0|jr$t j dj |t|tjr0|jr$t j dj |t|tjr0t|tjrt j dyy)zGLogs errors and returns bool indicating if transfer is valid for POSIX.z(Cannot preserve POSIX data from pipe: {}z#Cannot write POSIX data to pipe: {}z4Cannot preserve POSIX data for cloud-to-cloud copiesN)r)rFileUrl is_streamrrr/CloudUrl) source_urldestination_urls r 7-1rc4|s|r|jr||i|Sy)zEUseful for gating functions without repeating the below if statement.N)preserve_posix) posix_to_setuser_request_argsfunctionargskwargss rrun_if_setting_posixrJs&',=,L,L T $V $$ r)NN)F)NNF)5r4 __future__rrr collectionsrr<rg"googlecloudsdk.command_lib.storagerr,googlecloudsdk.command_lib.storage.resourcesrgooglecloudsdk.corer googlecloudsdk.core.cacher googlecloudsdk.core.utilr rcrfrirkrmrrrrrrrrr namedtupler7rBrQlrurWr\rvrwrrrrrr^rrrrr6rrrs`,&' 5:K#;.BBA& K' @'0,./,> 00N)+(():*8-)HJ=$ C 1%4&4 j9n)+((?A <   |~ F..7*  L(r