|NdZddlmZddlmZddlmZddlmZddlZddlZddlZddl Z ddl m Z ddl mZdd lmZddlm Z dd lmZdd lmZdd lmZdd lmZddlmZddlmZddlZdZdZGdde j<ZGdde j<Z ejBdgdZ"Gdde#Z$dZ%dZ&Gdde#Z'Gdde'Z(Gd d!e'Z)Gd"d#ejTZ+Gd$d%e#Z,y)&z/Manage and stream build logs from Cloud Builds.)absolute_import)division)print_function)unicode_literalsN) exceptions)cloudbuild_util)common)log) properties) resources)console_attr_os)requests)encodingz To live stream log output for this build, please ensure the grpc module is installed. Run: pip install grpcio and set: export CLOUDSDK_PYTHON_SITEPACKAGES=1 a The build is running, and logs are being written to the default logs bucket. This tool can only stream logs if you are Viewer/Owner of the project and, if applicable, allowed by your VPC-SC security policy. The default logs bucket is always outside any VPC-SC security perimeter. If you want your logs saved inside your VPC-SC perimeter, use your own bucket. See https://cloud.google.com/build/docs/securing-builds/store-manage-build-logs. ceZdZfdZxZS)NoLogsBucketExceptionc0d}tt| |y)Nz8Build does not specify logsBucket, unable to stream logs)superr__init__)selfmsg __class__s -lib/googlecloudsdk/api_lib/cloudbuild/logs.pyrzNoLogsBucketException.__init__<s DC /4__name__ __module__ __qualname__r __classcell__rs@rrr:s 55rrceZdZfdZxZS)4DefaultLogsBucketIsOutsideSecurityPerimeterExceptionc4tt| tyN)rr!r6DEFAULT_LOGS_BUCKET_IS_OUTSIDE_SECURITY_PERIMETER_TEXT)rrs rrz=DefaultLogsBucketIsOutsideSecurityPerimeterException.__init__Cs > OPrrrs@rr!r!AsQQrr!Response)statusheadersbodyceZdZdZdZdZy)RequestsLogTailerz9LogTailer transport to make HTTP requests using requests.c6tj|_yr#)creds_requests GetSessionsessionrs rrzRequestsLogTailer.__init__Ns!,,.DLrc2 |jjd|ddj|i}t|j|j |j S#tjj$r}tjd|zd}~wwxYw)NGETRangez bytes={0}-)r'zFailed to connect: %s) r.requestformatr% status_coder'contentrrRequestExceptionapi_exceptionsCommunicationError)rurlcursorresponsees rRequestzRequestsLogTailer.RequestQsK%% w (;(;F(CD&Fh h**H,<,N>N OO    / /K  - -.E.I JJKsAAB9BBN)rrr__doc__rr>rrr*r*KsA/Krr*c ddlm}|jS#t$r&tjj t YywxYw)zReturn a GCL LogTailer.rtailingN)googlecloudsdk.api_lib.loggingrC ImportErrorr outPrintLOG_STREAM_HELP_TEXT LogTailerrBs rGetGCLLogTailerrJZs@6     GGMM&' s,AAc|jr~|jjr)t|jjjS|jjr)t|jjj Sy)z/Separate CB4A requests to print different logs.F)optionsclusterboolname anthosCluster membership)builds rIsCB4ArSgs\ ]] }} %--'',, --  $ $ %----88 99 rc6eZdZdZdZdZdZdZefdZd dZ y) TailerBasez"Base class for log tailer classes.z REMOTE BUILD OUTPUT -ctjjjj }|rt j dd|S|S)z2Modify output for better screen reader experience.z---> )r VALUES accessibility screen_readerGetBoolresub)rtextr[s r_ValidateScreenReaderz TailerBase._ValidateScreenReadervs=%%33AAIIKM VVGR && Krcv|jr-|r*|jj|jyyy)z@Testing Hook: This method enables better verification of output.N)rFrGrstrip)rr_s r _PrintLogLinezTailerBase._PrintLogLine}s( xxD hhnnT[[]#xrctj\}}|j|j||jy)zDPrint a pretty starting line to identify start of build output logs.Nr GetTermSizerccenterOUTPUT_LINE_CHARrrwidth_s r_PrintFirstLinezTailerBase._PrintFirstLines3**,HE1szz%)>)>?@rctj\}}|j|j||jdzy)z@Print a pretty ending line to identify end of build output logs. Nreris r_PrintLastLinezTailerBase._PrintLastLines8**,HE1szz%)>)>?$FGrN)rX) rrrr?LOG_OUTPUT_BEGINrhr`rcrlror@rrrUrUqs,*,$ !1A HrrUcleZdZdZdej dfdZeejfdZ dZ dZ dZ y) GCLLogTailerzBHelper class to tail logs from GCL, printing content as available.NFct|_||_||_||_||_d|_||_d|_||_ y)NF) rJtailerbuild_id project_id timestamprFbuffer_window_secondslog_urlstopis_cb4a)rbuildId projectIdrxlogUrlrFr|s rrzGCLLogTailer.__init__sG"#DKDMDODNDH!"DDLDIDLrc ||j|j|j|j|t |S)zBuild a GCLLogTailer from a build resource. Args: build: Build resource, The build whose logs shall be streamed. out: The output stream to write the logs to. Returns: GCLLogTailer, the tailer of this build's logs. )r}r~rxrrFr|)idr~ createTimerrS)clsrRrFs r FromBuildzGCLLogTailer.FromBuilds9 //""|| u  rcv|jsy|jrydj|j}dj|j|j}|j rdj|j}|jj |g||j}|j|D].}|j|j}|j|0|jd |jr+|jd j|j y) z9Tail the GCL logs and print any new bytes to the console.Nprojects/{project_id}rwzslogName="projects/{project_id}/logs/cloudbuild" AND resource.type="build" AND resource.labels.build_id="{build_id}")rwrvzdlabels."k8s-pod/tekton.dev/taskRun"="{build_id}" OR labels."k8s-pod/tekton_dev/taskRun"="{build_id}")rv)ryz( BUILD FINISHED; TRUNCATING OUTPUT LOGS z"Logs are available at [{log_url}].)rz)rur{r4rwrvr|TailLogsryrlr` text_payloadrcrorzrparent log_filter output_logsoutputr_s rTailzGCLLogTailer.Tails, ;;  yy $ + +t + GF:;A6!%4==="{timestamp}" AND resource.labels.build_id="{build_id}")rwrxrvz(labels."k8s-pod/tekton.dev/taskRun"="{build_id}" OR labels."k8s-pod/tekton_dev/taskRun"="{build_id}") AND timestamp>="{timestamp}")rvrxasc)rorder_byrN) r4rwrxrvr|r FetchLogsrlr` textPayloadrcrors rrGzGCLLogTailer.Prints $ + +t + GF 028nn]]282$  || /06!%$..171J ""f>K   ' '(:(: ;d  r) rrrr?r r&r classmethodrFrrrrGr@rrrrrrsDJ :: " #$& P rrrczeZdZdZdZej fdZdZe ejfdZ d dZ dZ dZd Zy ) GCSLogTailerzBHelper class to tail a GCS logfile, printing content as available.z (possibly incomplete) ct|_|j|||_t j d|jzd|_||_d|_y)NzGCS logfile url is rF) r* transport _StorageUrlr:r debugr;rFr{)rbucketobjrFs rrzGCSLogTailer.__init__ sK&(DN,DHII#dhh./DKDHDIrcd}tjjjj rd}|j ||S)Nz-https://storage.googleapis.com/{bucket}/{obj}z2https://storage.mtls.googleapis.com/{bucket}/{obj})rr)r rY context_awareuse_client_certificater\r4)rrr url_patterns rrzGCSLogTailer._StorageUrls@AK&&==EEGHk   V  55rc|js t|j}d}|j|r|t|d}d|vr|}d}n|j dd\}}|dz }dj ||j }||||S) aLBuild a GCSLogTailer from a build resource. Args: build: Build resource, The build whose logs shall be streamed. out: The output stream to write the logs to. Raises: NoLogsBucketException: If the build does not specify a logsBucket. Returns: GCSLogTailer, the tailer of this build's logs. zgs://N/rXz{object}log-{id}.txt)objectr)rrrF) logsBucketr startswithlensplitr4r)rrRrF log_stripped gcs_prefix log_bucketlog_object_dir log_objects rrzGCSLogTailer.FromBuilds    ! ####LJz*!#j/"23l ,jn%1%7%7Q%?"z>n'.. 88/J    rcd |jj|j|j}|j dk(rtjdy|j dk(r(tjd|r|jy|j dk(s|j dk(rtjdj|j t|j |jd k(r|j|xjt|jz c_tj|j}||j!|}|j#|j%d |r|jy|j d k(r3tj&d |r|j|j(y|j dk\r[|j dkrLtj&dj|j |r|j|j(yt+|j,}|j |d<t j.||j|j#tj $r|rYywxYw)aEPoll the GCS object and print any new bytes to the console. Args: is_last: True if this is the final poll operation. Raises: api_exceptions.HttpError: if there is trouble connecting to GCS. api_exceptions.CommunicationError: if there is trouble reaching the server and is_last=True. Niz3Reading GCS logfile: 404 (no log yet; keep polling)iz7Reading GCS logfile: 416 (no new content; keep polling)z0Reading GCS logfile: {code} (read {count} bytes))codecountrrniz2Reading GCS logfile: 429 (server is throttling us)iiXz&Reading GCS logfile: got {0}, retryingr&)rr>r:r;r8r9r&r rror4rr(rlrDecoder`rcrbwarningLOG_OUTPUT_INCOMPLETEdictr' HttpError)ris_lastresdecodedr's rPollzGCSLogTailer.PollEs  NN " "488T[[ 9c zzS iiEF  zzS iiIJ    zzSCJJ#- iiBIIzzSXXJ01    kkS]"k)g  ,,W5 -.    zzS kkFG  D667  zzSSZZ#- kk:AA#**MN  D667 3;;G GH  " "7CHHdhh ??q  , ,    s0JJ/.J/c|js2|jtjd|js2|jdy)z;Tail the GCS object and print any new bytes to the console.rTrN)r{rrrr/s rrzGCSLogTailer.Tails6ii iik jjmii IIdIrcd|_yr)r{r/s rrzGCSLogTailer.Stops DIrc(|jdy)zPrint GCS logs to the console.TrN)rr/s rrGzGCSLogTailer.PrintsIIdIrN)F)rrrr?rr r&rrrrFrrrrrGr@rrrrsOJ3&)jj6 #%%NE@N rrc(eZdZdZfdZdZxZS)ThreadInterceptorz'Wrapper to intercept thread exceptions.cFtt| ||_d|_yr#)rrrtarget exception)rrrs rrzThreadInterceptor.__init__s T+-DKDNrc |jy#tj$r5}|jdk(rt |_n ||_Yd}~yYd}~yd}~wtj $r}||_Yd}~yd}~wwxYw)Ni)rr8rr5r!rr9)rr=s rrunzThreadInterceptor.runsg  kkm  # # # NO   , ,dnns A=&AA=,A88A=)rrrr?rrrrs@rrrs/ rrcPeZdZdZ ddZdZdZejfdZ dZ y) CloudBuildClientzGClient for interacting with the Cloud Build API (and Cloud Build logs).Nc|xstj|_|xstj|_||_||_yr#)rGetClientInstanceclientGetMessagesModulemessages support_gclpolling_interval)rrrrrs rrzCloudBuildClient.__init__s> ?O==?DKC A A CDM"D,DrcL|jdk(rEtjjd|jt j |j}|jjj|jj|jS)zGet a Build message. Args: build_ref: Build reference. Expects a cloudbuild.projects.locations.builds but also supports cloudbuild.projects.builds. Returns: Build resource zcloudbuild.projects.buildsz$cloudbuild.projects.locations.builds) collection projectsId locationsIdbuildsId)rO) Collectionr REGISTRYCreater~rDEFAULT_REGIONrrprojects_locations_buildsGetr+CloudbuildProjectsLocationsBuildsGetRequest RelativeName)r build_refs rGetBuildzCloudBuildClient.GetBuilds!==$$++;((%44<< ,!i ;; 0 0 4 4 AA'') B + ,,rc4tjjdt|jzdz|j|vr?|j |}t j|j|j|vr?|r|j|S)aChecks whether a log tailer should be stopped. Args: build: Build object, containing build status build_ref: Build reference, The build whose logs shall be streamed. log_tailer: Specific log tailer object working_statuses: Valid working statuses that define we should continue tailing Returns: Build message, the completed or terminated build. z1Waiting for build to complete. Polling interval: z second(s).) r r&rGstrrrrrr)rrRr log_tailerworking_statusess rShouldStopTailerz!CloudBuildClient.ShouldStopTailersJJH../02?@A ,,* *mmI&e jj&&' ,,* *oo Lrc|j|}|jr|jj|jjj j |jjj j|jjj jfvrtj||}n|jj|jjj j|jjj jfvr\|jrPtjdj|jjtj||}n:tjdj|jjd}|jj j"}|j$|j&g}d}|r&t)|j*}|j-|j/||||}|r(|j1|j2 |j2|S)aStreams the logs for a build if available. Regardless of whether logs are available for streaming, awaits build completion before returning. Args: build_ref: Build reference, The build whose logs shall be streamed. out: The output stream to write the logs to. Raises: NoLogsBucketException: If the build is expected to specify a logsBucket but does not. Returns: Build message, the completed or terminated build. )rFz7Streaming logs from GCL: requested logging mode is {0}.z2Not streaming logs: requested logging mode is {0}.N)r)rrLloggingr BuildOptionsLoggingValueValuesEnumNONESTACKDRIVER_ONLYCLOUD_LOGGING_ONLYrrrr infor4rrBuildStatusValueValuesEnumQUEUEDWORKINGrrstartrjoinr)rrrFrRrstatusesrts rStreamzCloudBuildClient.Streams" MM) $E ==EMM11 ""99>> ""99JJ ""99LL:  ))%S)9j    ""99JJ ""99LL#     hhHOO --  !"))%S)9j hhCJJ --  !"j}}""88H A :?? 3aggi  ! !%J"2 4Effh  kk Lrc|j|}|jr|jj|jjj j |jjj j|jjj jfvrtj|}n|jj|jjj j|jjj jfvrNtjdj|jjtj|}n:tjdj|jjd}|r|jyy)zPrint the logs for a build. Args: build_ref: Build reference, The build whose logs shall be streamed. Raises: NoLogsBucketException: If the build does not specify a logsBucket. z6Printing logs from GCL: requested logging mode is {0}.z.Logs not available: build logging mode is {0}.N)rrLrrrrrrrrrr rr4rrrG)rrrRrs rPrintLogzCloudBuildClient.PrintLog+sN MM) $E ==EMM11 ""99>> ""99JJ ""99LL:  ))%0j    ""99JJ ""99LL#  hhGNN --  !"))%0j hh?FF --  !"jr)NNFr) rrrr?rrrr rFrrr@rrrrs6O ! -,.0#&''5nrr)-r? __future__rrrr collectionsr] threadingrapitools.base.pyrr8!googlecloudsdk.api_lib.cloudbuildrrDr googlecloudsdk.corer r r googlecloudsdk.core.consoler googlecloudsdk.core.credentialsrr,googlecloudsdk.core.utilrrHr$Errorrr! namedtupler%rr*rJrSrUrrrThreadrrr@rrr s6&%'  9=1*#*)7F-:65J,,5Q:;K;KQ "; ! !*.K L K K HH:w:wtV:Vr ((.QvQr