kdZddlmZddlmZddlmZddlZddlZddlZddlm Z ddl m Z ddl mZdd l mZdd l mZdd l mZdd l mZdd l mZddl mZddl mZddlmZddlmZdd lmZddlmZddlm Zddl m!Z!ddl"m Z#ddl$m%Z%ddl&m'Z'ddl(m Z ddl(m)Z)ddl(m*Z*ddl(m+Z+ddl,m-Z-ddl.m/Z/ddl.m0Z1ddl2m3Z3ddl2m4Z4ddl5Z5ddl6m7Z7ddl6m8Z8d Z9d!Z:d"Z;d#Zd&Z?d'Z@Gd(d)e jZAGd*d+eAZBGd,d-eAZCGd.d/e jZDGd0d1e jZEGd2d3e jZFGd4d5e jZGGd6d7e jZHd8ZId9ZJd:ZKd;ZLd<ZMd=ZNd>ZOejjd?d?fd@ZRdAZSdBZTdCZUdDZVdddde%jdEfdFZXdGZYy)Hz+Utility methods used by the deploy command.)absolute_import)division)unicode_literalsN) exceptions) ext_runtime)appengine_api_client)build) cloud_build) docker_image) metric_names)runtime_builders)util) yaml_parsing)config) fingerprinter) enable_api) storage_util)appinfo) context_util)log)metrics) properties)progress_tracker)creds)store)files) platforms)filter)zipdefaultz-dot-?iXapUnable to verify that the Appengine Flexible API is enabled for project [{}]. You may not have permission to list enabled services on this project. If it is not enabled, this may cause problems in running your deployment. Please ask the project owner to ensure that the Appengine Flexible API has been enabled and that this account has permission to list enabled APIs.zNote: When deploying with a service account, the Service Management API needs to be enabled in order to verify that the Appengine Flexible API is enabled. Please ensure the Service Management API has been enabled on this project by the project owner.zEnabling the Appengine Flexible API failed on project [{}]. You may not have permission to enable APIs on this project. Please ask the project owner to enable the Appengine Flexible API on this project.ceZdZdZy)ErrorzBase error for this module.N__name__ __module__ __qualname____doc__5lib/googlecloudsdk/api_lib/app/deploy_command_util.pyr$r$^s#r+r$c eZdZy)PrepareFailureErrorN)r&r'r(r*r+r,r.r.bsr+r.c&eZdZdZdZfdZxZS)WindowMaxPathErrorzCRaised if a file cannot be read because of the MAX_PATH limitation.aThe following file couldn't be read because its path is too long: [{0}] For more information on this issue and possible workarounds, please read the following (links are specific to Node.js, but the information is generally applicable): * https://github.com/Microsoft/nodejstools/issues/69 * https://github.com/Microsoft/nodejs-guidelines/blob/master/windows-environment.md#max_path-explanation-and-workaroundsc^tt| |jj |y)N)superr0__init__ _WINDOWS_MAX_PATH_ERROR_TEMPLATEformat)selffilename __class__s r,r3zWindowMaxPathError.__init__vs( d, --44X>@r+)r&r'r(r)r4r3 __classcell__)r8s@r,r0r0fsK &"@@r+r0ceZdZdZy)DockerfileErrorzARaised if a Dockerfile was found along with a non-custom runtime.Nr%r*r+r,r;r;{sIr+r;ceZdZdZy)CloudbuildYamlErrorzFRaised if a cloudbuild.yaml was found along with a non-custom runtime.Nr%r*r+r,r=r=sNr+r=ceZdZdZy)CustomRuntimeFilesErrorzGRaised if a custom runtime has both a Dockerfile and a cloudbuild.yaml.Nr%r*r+r,r?r?sOr+r?ceZdZdZy)NoDockerfileErrorzNo Dockerfile found.Nr%r*r+r,rArAsr+rAceZdZdZy)UnsatisfiedRequirementsErrorz.Raised if we are unable to detect the runtime.Nr%r*r+r,rCrCs6r+rCctjjtjj|tj }tjjtjj|t jj}|jdk(r|rD|rBtdjtj t jj||r&tjdtj |y|r%tjdtj ytd|r/tdj|j |j|r/t#d j|j |jtjd |jy) aReturns True if the given directory needs a Dockerfile for this app. A Dockerfile is necessary when there is no Dockerfile in source_dir, regardless of whether we generate it here on the client-side, or in Cloud Container Builder server-side. The reason this function is more complicated than that is that it additionally verifies the sanity of the provided configuration by raising an exception if: - The runtime is "custom", but no Dockerfile is present - The runtime is not "custom", and a Dockerfile or cloudbuild.yaml is present - The runtime is "custom", and has both a cloudbuild.yaml and a Dockerfile. (The reason cloudbuild.yaml is tied into this method is that its use should be mutually exclusive with the Dockerfile.) Args: info: (googlecloudsdk.api_lib.app.yaml_parsing.ServiceYamlInfo). The configuration for the service. source_dir: str, the path to the service's source directory Raises: CloudbuildYamlError: if a cloudbuild.yaml is present, but the runtime is not "custom". DockerfileError: if a Dockerfile is present, but the runtime is not "custom". NoDockerfileError: Raised if a user didn't supply a Dockerfile and chose a custom runtime. CustomRuntimeFilesError: if a custom runtime had both a Dockerfile and a cloudbuild.yaml file. Returns: bool, whether Dockerfile generation is necessary. customzcA custom runtime must have exactly one of [{}] and [{}] in the source directory; [{}] contains bothzUsing %s found in %sFz7Not using %s because cloudbuild.yaml was found instead.TzYou must provide your own Dockerfile when using a custom runtime. Otherwise provide a "runtime" field with one of the supported runtimes.a%There is a Dockerfile in the current directory, and the runtime field in {0} is currently set to [runtime: {1}]. To use your Dockerfile to build a custom runtime, set the runtime field to [runtime: custom]. To continue using the [{1}] runtime, please remove the Dockerfile from this directory.a4There is a cloudbuild.yaml in the current directory, and the runtime field in {0} is currently set to [runtime: {1}]. To use your cloudbuild.yaml to build a custom runtime, set the runtime field to [runtime: custom]. To continue using the [{1}] runtime, please remove the cloudbuild.yaml from this directory.z.Need Dockerfile to be generated for runtime %s)ospathexistsjoinr DOCKERFILEr ResolverCLOUDBUILD_FILEruntimer?r5rinforAr;filer=)rN source_dirhas_dockerfilehas_cloudbuilds r,_NeedsDockerfilerSswF77>>ggll:v0013.77>>ggll:/88HHIK. \\X. # 2396  "2";";"K"K4   hh%v'8'8*E   hhH  "       78>vdii>Bll8L  MM   CDJ6iiD'  ((HH =t||L r+c^|jxr|j|j|S)aReturns whether we whould use runtime builders for this application build. If there is no image that needs to be built (service.RequiresImage() == False), runtime builders are irrelevant, so they do not need to be built. If there is an image that needs to be built, whether to use runtime builders is determined by the RuntimeBuilderStrategy, based on the service runtime and whether the service being deployed has a Dockerfile already made, or whether it needs one built. Args: service: ServiceYamlInfo, The parsed service config. strategy: runtime_builders.RuntimeBuilderStrategy, the strategy for determining whether a runtime should use runtime builders. needs_dockerfile: bool, whether the Dockerfile in the source directory is absent. Returns: bool, whether to use the runtime builders. Raises: ValueError: if an unrecognized runtime_builder_strategy is given ) RequiresImageShouldUseRuntimeBuildersrM)servicestrategyneeds_dockerfiles r,rVrVs10    ! O  + +GOO=M NPr+c0tj|jd}tj||}|r5|j }|Dcic]}|j |jc}Stdj|jcc}w)aReturns map of in-memory Docker-related files to be packaged. Returns the files in-memory, so that we don't have to drop them on disk; instead, we include them in the archive sent to App Engine directly. Args: info: (googlecloudsdk.api_lib.app.yaml_parsing.ServiceYamlInfo) The service config. dockerfile_dir: str, path to the directory to fingerprint and generate Dockerfiles for. Raises: UnsatisfiedRequirementsError: Raised if the code in the directory doesn't satisfy the requirements of the specified runtime type. Returns: A dictionary of filename relative to the archive root (str) to file contents (str). T)rdeployzYour application does not satisfy all of the requirements for a runtime of type [{0}]. Please correct the errors and try again.) rParamsparsedrIdentifyDirectoryGenerateConfigDatar7contentsrCr5rM)rNdockerfile_dirparams configurator dockerfilesds r,_GetDockerfilesrfs(   dkk$ ?&00H,113K,7 8KqAJJ "K 88 &  % '' 9sBci}d} tj|} tj|}tjtj||tj <|S#tj$r@}tj|j tj ||cYd}~Sd}~wwxYw#t$r?}tj|j tj |Yd}~|Sd}~wwxYw)zGets source context file information. Args: source_dir: str, path to the service's source directory Returns: A dict of filename to (str) source context file contents. zCould not generate [{name}]: {error} Stackdriver Debugger may not be configured or enabled on this application. See https://cloud.google.com/debugger/ for more information.)nameerrorN) rCalculateExtendedSourceContextsGenerateSourceContextErrorrrNr5CONTEXT_FILENAMEBestSourceContextsix text_typejsondumpsKeyError)rPsource_contextsmcontextsecontexts r,_GetSourceContextsForUploadrxs/!;;JGH D,,X6G58]] 76OL112   0 0HHQXX<88X BC  DHHQXX<88X BCC Ds5A,AC,B??5B:4B?:B? D  4DD cb|jd}t|dk(r|ddfS|d|dfS)z)Returns tuple (displayed app id, domain).:rN)splitlen) project_idls r,_GetDomainAndDisplayIdr7s=s!Vq[ Q4: 1qtr+ct|\}}|rtjntjj |||||S)z5Returns image tag according to App Engine convention.) gcr_domaindisplaydomainrWversion)rrDOCKER_IMAGE_NAME_DOMAIN_FORMATDOCKER_IMAGE_NAME_FORMATr5)projectrWrrrrs r, _GetImageNamer?sF*73/'64:& 0 0..# 171r+cxtj||r9tjj ||}t j ||s|Stj|}tjj|jj}dj|}|||<|S)aReturns the yaml path, optionally updating gen_files. Args: source_dir: str, the absolute path to the root of the application directory. service_path: str, the absolute path to the service YAML file skip_files: appengine.api.Validation._RegexStr, the validated regex object from the service info file. gen_files: dict, the dict of files to generate. May be updated if a file needs to be generated. Returns: str, the relative path to the service YAML file that should be used for build. )startz _app_{}.yaml) rIsDirAncestorOfrFrGrelpathr ShouldSkipReadFileContentsChecksum AddContentsencode HexDigestr5)rP service_path skip_files gen_filesrel_path yaml_contentschecksumgenerated_paths r, _GetYamlPathrKs :|4ww|:>H ??:x 0 o((6- ^^  ) )-*>*>*@ A K K M(!((2.+)N r+Fc  t||} t||| } |jsytjj dj |jtt|} | r| s| jt||tj|t||j||dtj } t#j$t&j(t*j,j/|| j0}t3||j4|j6j8| } t;j<|||| t#j$t&jP| rtSjT||| }tjVd|jX|j[t]j^|}| j0|d}| rEtajb|j6d }tajb|j6d r||d <|je|}nt;jf| j0}t;jh||}tk|| ||S#t>t@f$ra}tBjDjGr=|jHr1tK|jHtLkDrtO|jHd}~wwxYw) aBuilds and pushes a set of docker images. Args: project: str, The project being deployed to. service: ServiceYamlInfo, The parsed service config. upload_dir: str, path to the service's upload directory source_files: [str], relative paths to upload. version_id: The version id to deploy these services under. code_bucket_ref: The reference to the GCS bucket where the source will be uploaded. gcr_domain: str, Cloud Registry domain, determines the physical location of the image. E.g. `us.gcr.io`. runtime_builder_strategy: runtime_builders.RuntimeBuilderStrategy, whether to use the new CloudBuild-based runtime builders (alternative is old externalized runtimes). parallel_build: bool, if True, enable parallel build and deploy. use_flex_with_buildpacks: bool, if true, use the build-image and run-image built through buildpacks. Returns: BuildArtifact, Representing the pushed container image or in-progress build. Raises: DockerfileError: if a Dockerfile is present, but the runtime is not "custom". NoDockerfileError: Raised if a user didn't supply a Dockerfile and chose a custom runtime. UnsatisfiedRequirementsError: Raised if the code in the directory doesn't satisfy the requirements of the specified runtime type. ValueError: if an unrecognized runtime_builder_strategy is given Nz2Building and pushing image for service [{service}])rWF)rareponocachetag)rzUsing runtime builder [%s]) _OUTPUT_IMAGE_GAE_APPLICATION_YAML_PATHpython_version_GOOGLE_RUNTIME_VERSION)6rSrVrUrstatusPrintr5moduledictrxupdaterfr ImagerrDOCKER_IMAGE_TAGrCustomTimedEventr CLOUDBUILD_UPLOAD_STARTrObjectReference FromBucketRef tagged_reporrOr]rr UploadSourceOSErrorIOErrorrOperatingSystem IsWindowsr7r}_WINDOWS_MAX_PATHr0CLOUDBUILD_UPLOADr FromServiceInforNbuild_file_uriWarnIfDeprecatedrConvertToPosixPathrGetRuntimeConfigAttrLoadCloudBuildGetDefaultBuild FixUpBuild _SubmitBuild)rrW upload_dir source_files version_idcode_bucket_refrruntime_builder_strategyparallel_builduse_flex_with_buildpacksrYuse_runtime_buildersrimage object_refrelative_yaml_patherrbuilder_reference yaml_path substituterr s r,BuildAndPushDockerImagerfsT&gz:1'2J2BD    **: vgnnv%'.z:;)2_Wj9:    '..*j I  ! !  #%  <??@++99u((**#J $+NN$=$=yJ Zz'02 <99:(8857HH )+<+K+KL&&(''(:;I**&/J #88 ..*,n  * *7>>;K L0> ,-  , ,Z 8E  ' '(9(9 :E   3% eUG^ <<; 7    **, #cll+.?? ..  sJL"AK>>Lctjtjjj j }|r4|tkDr+d}tjdj|t|rgtjtjtj j#||}t$j&j)|Stjtj*tj j-||tjtj.t$j&j1|j2S)a[Builds and pushes a set of docker images. Args: build: A fixed up Build object. image: docker_image.Image, A docker image. project: str, The project being deployed to. parallel_build: bool, if True, enable parallel build and deploy. Returns: BuildArtifact, Representing the pushed container image or in-progress build. FzProperty cloud_build_timeout configured to [{0}], which exceeds the maximum build time for parallelized beta deployments of [{1}] seconds. Performing serial deployment.)r)r GetServiceTimeoutSecondsrVALUESappcloud_build_timeoutGetMAX_PARALLEL_BUILD_TIMErrNr5rrr CLOUDBUILD_EXECUTE_ASYNC_STARTcloudbuild_buildCloudBuildClientExecuteCloudBuildAsync app_build BuildArtifactMakeBuildIdArtifactFromOpCLOUDBUILD_EXECUTE_STARTExecuteCloudBuildCLOUDBUILD_EXECUTEMakeImageArtifactr)r rrr build_timeoutbuild_ops r,rrs66//3357-}'>>NHH 117 2245  \HHI002II wJ H  " " < r@s 2&' >';92337+34=G6F7H07*#'*81<*.     ,-N $J  $%@@*Jj&&JO***OPj..P ((7:#3#37L^P8'@: F.DDJJ"h=V HF5(-C`:D$"..tN;b r+