@GdZddlmZddlmZddlmZddlZddlZddlZddlZddl m Z ddl m Z ddl m Z dd l mZdd l mZdd lmZdd lmZdd lmZddlmZegdZegdZej2eegZddddddddZdddddddZej:j<ej>ej:j@ejBiZ"ej:j<ejFej:j@ejHiZ%ejLd Z'd!Z(d"Z)d#Z*d$Z+d%Z, d3d&Z-d'Z.d(Z/d)Z0d*Z1d+Z2d,Z3 d4d-Z4d.Z5d/Z6d0Z7 d4d1Z8 d4d2Z9y)5z,Tools for making the most of S3Api metadata.)absolute_import)division)unicode_literalsN) metadata_util)xml_metadata_field_converters)errors) storage_url)user_request_args_factory)gcs_resource_reference)resource_reference)s3_resource_reference)log) CacheControlContentDispositionContentEncodingContentLanguage ContentTypeMetadata)ACLAccessControlPolicy StorageClass cache_controlcontent_dispositioncontent_encodingcontent_language content_typemd5_hash storage_class)rrrrr ContentMD5rzauthenticated-readzbucket-owner-full-controlzbucket-owner-readprivatez public-readzpublic-read-write)authenticatedReadbucketOwnerFullControlbucketOwnerReadr publicReadpublicReadWritez^[a-fA-F0-9]{32}$ctj|}|jdd|js|S|jj}|rD|D]?}tj|}|jd}|dj ||dA|jj }|rt|}g} |jdgD]r} | jdi} t| jd| jd| jd f} |j| rb| j | t| |d<|S) aWReturns full ACL policy object with requested changes. Args: acl_dict (dict): Contains S3-format ACL policy dict for bucket or object. Usually of the form: "{"Grants": [...], "Owner": {...}}". See: https://boto3.amazonaws.com/v1/documentation/api/latest/reference /services/s3.html#S3.Client.get_bucket_acl request_config (request_config_factory._RequestConfig): Contains desired changes for the ACL policy. Returns: dict: Deep copy of acl_dict with added and removed grants and removed "ResponseMetadata" field to allow for reuse in PUT API calls. ResponseMetadataN PermissionGrants)r(Granteer* EmailAddressIDURI) copydeepcopypop resource_argsacl_grants_to_addappendacl_grants_to_removesetget intersection) acl_dictrequest_config acl_dict_copyr2 new_grantnew_grant_copy permissionr4entity_identifiers_to_removefiltered_grantsexisting_grantexisting_granteeexisting_grantee_identifierss 7lib/googlecloudsdk/api_lib/storage/xml_metadata_util.py,get_acl_policy_with_added_and_removed_grantsrDUsL --)-&-  % % $22DD& }}Y/n!%%l3jH$$"#& '(55JJ#&';#< O'++Hb9'++Ir:%(    /1A1E1Ed1K    & (&)"* 6 6 &(~.:.M( c>|si}|s|S|D]}||vs||||<|S)zECopy fields(provided in arguments) from one metadata dict to another.)source_metadata_dictdestination_metadata_dictfieldsfields rC_copy_metadatarLs? " "  $$e $$)=e)D& #"rEc$t||tS)z:Copies common S3 fields from one metadata dict to another.)rL_COMMON_S3_METADATA_FIELDSrHrIs rCcopy_object_metadatarPs   rEc$t||tS)z>Copies user_metadata fields from one metadata dict to another.)rL_USER_METADATA_FIELDSrOs rCcopy_user_metadata_fieldsrSs  rEcl|tvr$tjdj|t|S)aTranslates Apitools predefined ACL enum key (as string) to S3 equivalent. Args: predefined_acl_string (str): Value representing user permissions. Returns: Translated ACL string. Raises: Error: Predefined ACL translation could not be found. zACould not translate predefined_acl_string {} to AWS-accepted ACL.)*_GCS_TO_S3_PREDEFINED_ACL_TRANSLATION_DICTrErrorformat)predefined_acl_strings rC%translate_predefined_acl_string_to_s3rYs<"LL ,, #V$9:  44I JJrEcRtj||||jdS)ajCreates storage_url.CloudUrl from S3 API response. Args: scheme (storage_url.ProviderPrefix): Prefix used for provider URLs. object_dict (dict): Dictionary representing S3 API response. bucket_name (str): Bucket to include in URL. object_name (str | None): Object to include in URL. Returns: storage_url.CloudUrl populated with data. VersionId)scheme bucket_name resource_name generation)r CloudUrlr6)r\ object_dictr] object_names rC!_get_object_url_from_xml_responsercs,    -  //rEcd|vr|jd}nd|vr|djd}nd}|r3|jdr"|jdr|jdS|S)z.Returns the cleaned-up etag value, if present.ETagCopyObjectResultN")r6 startswithendswithstrip)raetags rC _get_etagrlsi { ??6 "D[( ) * . .v 6D D dooc"t}}S'9 ::c? +rEc|rOtj|r:tjt j |}|j dStjd||y)z6Returns base64 encoded MD5 hash, if etag is valid MD5.zutf-8)encodingzSNon-MD5 etag ("%s") present for object: %s. Data integrity checks are not possible.N) MD5_REGEXmatchbase64 b64encodebinascii unhexlifydecoderdebug)rk object_url encoded_bytess rC_get_md5_hash_from_etagrys[ iood#$$X%7%7%=>M     11II 348*F rEc|jdi}|jdi}|jddj}tjd|}|r|j dSy)z@Returns base64 encoded CRC32C hash from object response headers.r' HTTPHeadersz x-goog-hashzcrc32c\=([^,]+)N)r6rjresearchgroup)raresponse_metadataheaders hash_headerresults rC!_get_crc32c_hash_from_object_dictrsf!oo&8"=  ! !- 4' M2.446+ 99' 5& <<? rEctt|tr'd|vr#|j}|jd|S|S)z?Returns the error string if value is error or the value itself.r') isinstancedictr.r0)value value_copys rC_get_error_or_valuers7t!3u!<JNN%&  ,rEcRt|jd}|dk(rd}n|dk(rd}t|jd}t|tr|jddk(rd}nd }t |t j ||t|jd t|jd t|jd t|jd |t|jd||t|jd S)aVCreates resource_reference.S3BucketResource from S3 API response. Args: scheme (storage_url.ProviderPrefix): Prefix used for provider URLs. bucket_dict (dict): Dictionary representing S3 API response. bucket_name (str): Bucket response is relevant to. Returns: resource_reference.S3BucketResource populated with data. Payer RequesterT BucketOwnerF VersioningStatusEnabledNr CORSRulesLifecycleConfigurationLoggingEnabledLocationConstraintWebsite) acl cors_configlifecycle_configlogging_configrequester_payslocationmetadataversioning_enabledwebsite_config)rr6rr_SCHEME_TO_BUCKET_RESOURCE_DICTr r`)r\ bucket_dictr]rrs rC%get_bucket_resource_from_xml_responsers'{w'?@.{"N&N*;??<+HI"D)h'94 ( 06;/ kooe4 5%kook&BC* //2 35(9I)JK#";??3G#HI+()CD F FrEcd|jdd}|sy|Dcic] }|d|d c}Scc}w)zHParse object tags from object response in format of GCS object contexts.TagSetNKeyValue)r6)raobj_tagstags rC_parse_object_tags_if_anyr'sJ__Xt ,(  +3 *23c%j#g,(  s-c t||||xs|d}d|vr|jd}n|jd}|jd|jd}t|}|r|} n|jd} | r| |d<t| } t ||fid| d|jd d |jd d |jd d|jdd|jdd|jdd|jdd|jdd|d|d|jddt ||d|d|d|jd d!|jd} |t jjk(rt|| _ |t jjk(rt|| _ | S)"aCreates resource_reference.S3ObjectResource from S3 API response. Args: scheme (storage_url.ProviderPrefix): Prefix used for provider URLs. object_dict (dict): Dictionary representing S3 API response. bucket_name (str): Bucket response is relevant to. object_name (str|None): Object if relevant to query. acl_dict (dict|None): Response from S3 get_object_acl API call. Returns: resource_reference.S3ObjectResource populated with data. rSize ContentLengthServerSideEncryptionSSECustomerAlgorithmrrrrcomponent_count PartsCountrrrrrrrr creation_time LastModified custom_fieldsrencryption_algorithmrkkms_key SSEKMSKeyIdrrsizerr update_time)rcr6rlr_SCHEME_TO_OBJECT_RESOURCE_DICTryr ProviderPrefixGCSr crc32c_hashS3rtags) r\rar]rbr8rwrrrk raw_acl_datarobject_resources rC%get_object_resource_from_xml_responser5s"1 k; (I{57IK* { ??6 "D ??? +D$koo.DEG ; $ L??5)L%KL)#3F;  OON3"ool3  &//*>?  #'89 #'89??=1 OON3 OOJ/0 oom,'tZ8 !" OON3#$//.1%/* {))---"CK"PO {)),,,4[AO rEcf|d}tjtj||||S)aBCreates resource_reference.PrefixResource from S3 API response. Args: scheme (storage_url.ProviderPrefix): Prefix used for provider URLs. prefix_dict (dict): The S3 API response representing a prefix. bucket_name (str): Bucket for the prefix. Returns: A resource_reference.PrefixResource instance. Prefix)r\r]r^)prefix)r PrefixResourcer r`)r\ prefix_dictr]rs rC%get_prefix_resource_from_xml_responserys= x &  * *!   rEcDi}|j}|r|j.|jtj|j|j .|jtj |j |j.|jtj|j|j|j|d<|j.|jtj|j|j.|jtj|j|j |j9|jtj |j|j|S)z>Returns S3 bucket metadata dict fields based on RequestConfig.r)r1cors_file_pathupdater process_corslabels_file_pathprocess_labelslifecycle_file_pathprocess_lifecyclerrprocess_requester_pays versioningprocess_versioningweb_error_pageweb_main_page_suffixprocess_website)r9rr1s rC,get_bucket_metadata_dict_from_request_configrss ( ..-##/oo ' 4 4** ,-%%1oo ' 6 6,, ./((4oo ' 9 9// 12)'4'='=h#$##/oo ' > >** ,-+oo ' : :&& () $$0**6oo ' 7 7**M,N,N PQ /rEcF|tjk(rd||<y||||<yy)z)Sets appropriate metadata based on value.N)r CLEAR)rkeyrs rC_process_value_or_clear_flagrs. '---HSM HSMrEc|j}|y|jytjD]}t ||dsyt j ||j|j||S)z>Checks the presence of user_metadata fields in request_config.NFTattributes_resource known_posix) r1rX_S3_TO_GENERIC_FIELD_NAMES_valuesgetattrrhas_updated_custom_fieldspreserve_posixpreserve_symlinks)r9rrr1rs rC0is_user_metadata_field_present_in_request_configrsy !..- ))5 *113e}eT* 4  0 0##&&-  rEc 6|jt|j|d<|j}|jdi}t j ||||}|||d<|r5t jD]\}}t||t||dyy)aReturns S3 object metadata dict fields based on RequestConfig. Args: object_metadata (dict): Existing object metadata. request_config (request_config): May contain data to add to object_metadata. attributes_resource (Resource|None): If present, used for parsing POSIX and symlink data from a resource for the --preserve-posix and/or --preserve_symlink flags. This value is ignored unless it is an instance of FileObjectResource. posix_to_set (PosixAttributes|None): Set as custom metadata on target. Nrrr) rXrYr1r6rget_updated_custom_fieldsritemsrr) object_metadatar9r posix_to_setr1existing_metadatacustom_fields_dictrKrs rC/update_object_metadata_dict_from_request_configrs$))5B,,.OE!..-%))*b9$>>-  #"4OJ399; u"    <rE)N)NN):__doc__ __future__rrrrqrsr.r~googlecloudsdk.api_lib.storagerr"googlecloudsdk.command_lib.storagerr r ,googlecloudsdk.command_lib.storage.resourcesr r r googlecloudsdk.corer frozensetrR_NON_USER_METADATA_FIELDSunionrNrrUrrGcsBucketResourcerS3BucketResourcerGcsObjectResourceS3ObjectResourcercompilerorDrLrPrSrYrcrlryrrrrrrrrrrrGrErCrs3&'  8H5:HOKN#!#&' /Y[..67 $/))!#.9** .*""$:$L$L!!#8#I#I# ""$:$L$L!!#8#I#I# BJJ+ , 2j #K.37/,  #FL "7;37 AH("J: -rE