$dZddlmZddlmZddlmZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZddlZdd lmZddgZdZdZdZddZdZdZGdde Z!dZ"dZ#dZ$y)zEUtility methods to upload source to GCS and call Cloud Build service.)absolute_import)division)unicode_literalsN)encoding)cloudbuild_util) storage_api)log) properties)files)times)filter Dockerfile .dockerignorecFtjj|}tjd|}t |D]6}tjj ||}|j||d8tj|D]\}} tj| j} tj|} t| j| _|j#| | | j%|j%y)aCreate tarfile for upload to GCS. The third-party code closes the tarfile after creating, which does not allow us to write generated files after calling docker.utils.tar since gzipped tarfiles can't be opened in append mode. Args: upload_dir: the directory to be archived gen_files: Generated files to write to the tar paths: allowed paths in the tarfile gz: gzipped tarfile object wmodefileobjF)arcname recursive)name)rN)ospathabspathtarfileopensortedjoinaddsix iteritemsioBytesIOencodeTarInfolengetvaluesizeaddfileclose) upload_dir gen_filespathsgzroottr full_pathrcontents genfileobjtar_infos -lib/googlecloudsdk/api_lib/app/cloud_build.py _CreateTarr6.s  $$ llR(!Umd T4(IEE)TUE3 i0ndHHOO-.JD)H ++-.HMIIh I+ 1 '')c`tjj|d}t}d}tjj |rt j |}n|jd}|r9ttt|j}|ttz}|S)a Helper function to read the .dockerignore on disk or in generated files. Args: upload_dir: the path to the root directory. gen_files: dict of filename to contents of generated files. Returns: Set of exclusion expressions from the dockerignore file. rN) rrrsetexistsr ReadFileContentsgetr bool splitlinesBLOCKLISTED_DOCKERIGNORE_PATHS)r+r, dockerignoreexcludeignore_contentss r5_GetDockerignoreExclusionsrCIsj/:, E'/WW^^L!,,\:OmmO4O&99;<=G s122G .r7cddl}tjj|}|jj |t |}|j||S)aHelper function to filter paths in root using dockerignore and skip_files. We iterate separately to filter on skip_files in order to preserve expected behavior (standard deployment skips directories if they contain only files ignored by skip_files). Args: upload_dir: the path to the root directory. source_files: [str], relative paths to upload. exclude: the .dockerignore file exclusions. Returns: Set of paths (relative to upload_dir) to include. rN)dockerrrrutils exclude_pathslistintersection_update)r+ source_filesrArEr/r-s r5_GetIncludedPathsrKbsH"  $$ ,, $ $T4= 9%L) ,r7c|xsi}t||}t|||}tj5}tjt j j|d}tjd|5}t||||ddd|jtj} | j|j|dddy#1swYRxYw#1swYyxYw)aUpload a gzipped tarball of the source directory to GCS. Note: To provide parity with docker's behavior, we must respect .dockerignore. Args: upload_dir: the directory to be archived. source_files: [str], relative paths to upload. object_ref: storage_util.ObjectReference, the Cloud Storage location to upload the source tarball to. gen_files: dict of filename to (str) contents of generated config and source context files. zsrc.tgzwbrN)rCrKr TemporaryDirectoryBinaryFileWriterrrrgzipGzipFiler6r*r StorageClient CopyFileToGCSr) r+rJ object_refr,dockerignore_contentsincluded_pathstemp_dirfr.storage_clients r5 UploadSourcerZ~so2)4ZK$, 57. !X rww||Hi@AA D! ,Y; -GGI ..0N  4 "! , ,"!s%A C+?CAC+C( $C++C4c`|ytj|d}t|jS)zAReturns the service timeout in seconds given the duration string.Ns)default_suffix)r ParseDurationint total_seconds)timeout_property_strbuild_timeout_durations r5GetServiceTimeoutSecondsrcs5!  ../C>AC # 1 1 22r7cN|yt|}tj|dzS)zAReturns the service timeout duration string with suffix appended.Nr\)rcr text_type)rabuild_timeout_secss r5GetServiceTimeoutStringrgs,! /0DE ) *S 00r7c"eZdZdZfdZxZS)InvalidBuildErrorzFError indicating that ExecuteCloudBuild was given a bad Build message.cJtt| dj|y)NzeField [{}] was provided, but should not have been. You may be using an improper Cloud Build pipeline.)superri__init__format)selffield __class__s r5rlzInvalidBuildError.__init__s" T+ ==CVE]Lr7)__name__ __module__ __qualname____doc__rl __classcell__)rps@r5ririsNLLr7ricB|D]}t||dt|y)zDValidates that a Build message doesn't have fields that we populate.N)getattrri)buildfieldsros r5_ValidateBuildFieldsrzs&eueT". e $$r7c *tj}tjjj j }tjdj||j|j|dd|dgg|gS)aGet the default build for this runtime. This build just uses the latest docker builder image (location pulled from the app/container_builder_image property) to run a `docker build` with the given tag. Args: output_image: GCR location for the output docker image (e.g. `gcr.io/test-gae/hardcoded-output-tag`) Returns: Build, a CloudBuild Build message with the given steps (ready to be given to FixUpBuild). zUsing builder image: [{0}]rxz-t.)rargs)stepsimages) rGetMessagesModuler VALUESappcontainer_builder_imageGetr debugrmBuild BuildStep) output_imagemessagesbuilders r5GetDefaultBuildrs . . 0(    ! ! 9 9 = = ?')) ( / / 89    W&-t\3%G I J^  r7ctj}tj|}|jr9|jj j tjdt|dttjjjj|_|j"|_|j'|j)|j"|j*|_|S)aReturn a modified Build object with run-time values populated. Specifically: - `source` is pulled from the given object_ref - `timeout` comes from the app/cloud_build_timeout property - `logsBucket` uses the bucket from object_ref Args: build: cloudbuild Build message. The Build to modify. Fields 'timeout', 'source', and 'logsBucket' will be added and may not be given. object_ref: storage_util.ObjectReference, the Cloud Storage location of the source tarball. Returns: Build, (copy) of the given Build message with the specified fields populated. Raises: InvalidBuildError: if the Build message had one of the fields this function sets pre-populated key)r)sourcetimeout logsBucket)bucketobject) storageSource)rrrCopyProtoMessage substitutionsadditionalPropertiessortoperator attrgetterrzrgr rrcloud_build_timeoutrrrrSource StorageSourcerr)rxrTrs r5 FixUpBuildrs, . . 0(  # #E *%  ,,11    &2(uAB)//3357%-&&%**""+!%, ,r7)N)%rt __future__rrrrPr"rrrapitools.base.pyr!googlecloudsdk.api_lib.cloudbuildrgooglecloudsdk.api_lib.storagergooglecloudsdk.corer r googlecloudsdk.core.utilr r r six.movesr r?r6rCrKrZrcrg ValueErrorrirzrrr7r5rs"L&'  %=6#***  #/!@ 6285@31L L%0-r7