Y dZddlmZddlmZddlmZddlZddlmZddlmZ ddl m Z dd l m Zdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZdZddddddddddd Z d d!d"d#d$d%d&d'd(d)d Z!d*Z"d+Z#d,Z$d-Z% d9d.Z& d9d/Z'd0Z(d1Z)d2Z*d3Z+d4Z,d5Z-d6Z.d7Z/d8Z0y):z-Utilities for the model garden command group.)absolute_import)division)unicode_literalsN)encoding) exceptions) operations)client)metric) quota_info)endpoints_util) models_util)operations_util)log) properties)requests) resources) console_io?+CustomModelServingP4GPUsPerProjectPerRegion+CustomModelServingT4GPUsPerProjectPerRegion+CustomModelServingL4GPUsPerProjectPerRegion,CustomModelServingK80GPUsPerProjectPerRegion-CustomModelServingV100GPUsPerProjectPerRegion-CustomModelServingP100GPUsPerProjectPerRegion-CustomModelServingA100GPUsPerProjectPerRegion1CustomModelServingA10080GBGPUsPerProjectPerRegion-CustomModelServingH100GPUsPerProjectPerRegion+CustomModelServingV5ETPUPerProjectPerRegion) NVIDIA_TESLA_P4NVIDIA_TESLA_T4 NVIDIA_L4NVIDIA_TESLA_K80NVIDIA_TESLA_V100NVIDIA_TESLA_P100NVIDIA_TESLA_A100NVIDIA_A100_80GBNVIDIA_H100_80GBTPU_V5_LITEPOD#custom_model_serving_nvidia_p4_gpus#custom_model_serving_nvidia_t4_gpus#custom_model_serving_nvidia_l4_gpus$custom_model_serving_nvidia_k80_gpus%custom_model_serving_nvidia_v100_gpus%custom_model_serving_nvidia_p100_gpus%custom_model_serving_nvidia_a100_gpus*custom_model_serving_nvidia_a100_80gb_gpus%custom_model_serving_nvidia_h100_gpuscustom_model_serving_tpu_v5eametric.type="serviceruntime.googleapis.com/quota/allocation/usage" AND resource.type="consumer_quota" AND metric.label.quota_metric="aiplatform.googleapis.com/{}" AND resource.label.project_id="{}" AND resource.label.location="{}" AND resource.label.service="aiplatform.googleapis.com"ctjj||tjj j jddS)z >sC H  rBc|rd|d|jdddtSd|d|d|jdddtS)Nrwrxryrzz publishers-z-models-r{r}s r@GetOneClickEndpointLabelValuersq  * . 6 6sC @   n%Xj\;M:NO    % $  rBctjjd|d|d}|jdk7rt j dt |jdS)z9Checks if the HF model is gated or not by calling HF API."https://huggingface.co/api/models//z ?blobs=truezfSomething went wrong when we call HuggingFace's API to get the model metadata. Please try again later.gated)r GetSessionget status_codecore_exceptions InternalErrorbooljson)rr hf_responses r@IsHFModelGatedrsq##%))*>*:!J<{S+#  ' ' 3  k ) **rBctjjdd|d|ddd|i}|jdk7rt j d y) NGETrrz /auth-check AuthorizationzBearer )headersrzaThe Hugging Face access token is not valid or does not have permission to access the gated model.)rrrequestrrError)hf_tokenrrrs r@VerifyHFTokenPermissionrsr##%-- *>*:!J<{S'( 45.+ #    &  rBc |jjj}d}|j s|j s |jr|D]}|j r-|jjj|j k7s|j rDt|jjj|j jk7s/|jr$|jj|jk7r|}n|stjdt j"j%dn$|d}t j"j%d|jj}|jj}|jr,t j"j%d|j|jr,t j"j%d|j|j&r,t j"j%d |j&|r"t j"j%d ||S#t$rtjdwxYw) zGReturns a best suited deployment configuration for the publisher model.zModel does not support deployment, please use a deploy-able model instead. You can use the `gcloud ai model-garden models list` command to find out which ones are currently supported by the `deploy` command.NzThe machine type, accelerator type and/or container image URI is not supported by the model. You can use `gcloud alpha/beta ai model-garden models list-deployment-config` command to find the supported configurationsz,Using the selected deployment configuration:rz+Using the default deployment configuration:z Machine type: z Accelerator type: z Accelerator count: z Container image URI: )supportedActionsmultiDeployVertexAttributeErrorrr machine_typerKcontainer_image_uridedicatedResources machineSpec machineTypestracceleratorTypeupper containerSpecimageUrirstatusPrintacceleratorCount)argspublisher_model multi_deploy deploy_configdeploy machine_specrs r@GetDeployConfigrs& ((::LL- $//43K3K++77CC""###f//;;KKL&&,,./&&&&//43K3KK m '*   ! ! &  JJCD!OMJJBC11==,%33<<JJ|'?'?&@AB!!JJ*<+G+G*HIJ""JJ+L,I,I+JKLJJ-.A-BCD m      s I I?c B|dk(rd}d}n |dk(rd}d}tjjjj }t |j ||}||kr3tjdt|d|j d|d |d t|j ||}||z|kDr6tjdt|d|j d |d |d |d tjjd |d|j d|d |d y)z8Checks the accelerator quota for the project and region.zct5lp-hightpu-1tr(zct5lp-hightpu-4tz+The project does not have enough quota for z in z) to deploy the model. The quota limit is z and you are requesting for z. Please use a different region or request more quota by following https://cloud.google.com/vertex-ai/docs/quotas#requesting_additional_quota.z+ to deploy the model. The current usage is z out of zNThe project has enough quota. The current usage of quota for accelerator type z in region z is ryN)rr:r;r<r=rNrJrrrjrurrr)rrrKaccelerator_countr< quota_limitrts r@CheckAcceleratorQuotar s '''))'    " " * * 4 4 6't{{G5EF+$$    12B C DE KK=11< >,-.W  W !g7GH-&&4    12B C DE KK=33@/B =,-.W  W  **+,K }E  Q0rBc T|j|||jjj|jjjj d|g}t j ||tj|j}|tjdtj|}d|vrtjdtjj!dj#|d|dj%dd S) z)Creates a Vertex endpoint for deployment.z mg-cli-deploy)keyrI)additionalProperties)labelszEInternal error: Failed to create a Vertex endpoint. Please try again.namezOCreated Vertex AI endpoint: {}. Starting to upload the model to Model Registry.r) CreateBetamessages$GoogleCloudAiplatformV1beta1Endpoint LabelsValueAdditionalPropertyrWaitForOpMayber ParseOperationrrrrMessageToPyValuerrrrisplit) endpoint_name label_value region_refoperation_clientendpoints_clientcreate_endpoint_opcreate_endpoint_response_msgresponses r@CreateEndpointr8s2(22  & & K K W W''LLXXkk%[l X3 "1!?!?##$6$;$;<" ")  ' 'O  & &'C D( 8  ' 'O ** x  &    $R ((rBcjd\}}} |jjrM|jjD cic]} | j| j}} |rd|vr|j|d<|jj rt |jj }|jjrt |jj} tj} | j|jjj|dd|j|jj| |||jj dj"gd|jj$|jj&| j(j+| j(j-dj/|rd|zn|| } t1j2t5j6| t9j:| j } | t=j>d tAjB| }d |vrt=j>d tDjFjId j/|d |d jKddScc} w)z1Uploads the Model Garden model to Model Registry.)NNNHUGGING_FACE_HUB_TOKENNrzpublishers/{}/models/{}rw)publicModelName)modelGardenSource)base_model_source)operations_clientopop_refz`Internal error: Failed to upload a Model Garden model to Model Registry. Please try again later.modelzEUploaded model to Model Registry at {}. Starting to deploy the model.rr)&renvrrIhugging_face_access_tokenrlistcommand client_models ModelsClient UploadV1Beta1CONCEPTSrJr9 artifactUrirports containerPort predictRoute healthRouter0GoogleCloudAiplatformV1beta1ModelBaseModelSource-GoogleCloudAiplatformV1beta1ModelGardenSourcerirrrOperationsClientr ParseModelOperationrrrrrrrr)rrrequires_hf_tokenr~uploaded_model_namerpublisher_model_namecontainer_env_varscontainer_argscontainer_commandsvar models_clientupload_model_opupload_model_response_msgupload_model_responses r@ UploadModelrbs>,7%.(^&a `0/2.<<"335   , ,_-A-A B &  ' ' - #334MN ))  ' ' - ** $W-.  w ' - -c 22 66ysJ0c |jjj}|jjj}d} ||7i} |,t |j j dd| d<||| d<|jt||j||j||jjj| dd} tj|| tj| jd| jj!d d } t#d |d | d |jd|d |jd y)z8Deploys the Model Registry model to the Vertex endpoint.NrzrxtypecountT)raccelerator_dictenable_access_loggingenable_container_logging) asynchronousrrDeploying the model to the endpoint. To check the deployment status, you can try one of the following methods: 1) Look for endpoint `g` at the [Vertex AI] -> [Online prediction] tab in Cloud Console 2) Use `gcloud ai operations describe --region=zc` to find the status of the deployment long-running operation 3) Use `gcloud ai endpoints describe z+` command to check the endpoint's metadata.)rrrrrlowerraDeployModelBetarArJrrrr rrrprint) rrr>rmodel_idrrrKrrdeploy_model_op deploy_op_ids r@ DeployModelrso&&22BB&&22CC!%6%B#!$%5!6!!F!FsC!Pv$"3w$44[$++. kk 33??KK' #5 /  ##O$8$89  !%%++C04,!?#. 4;;-8(MDKK=A+ + rBc 4 |%|j}|j}|j}nd}d}d}|jd8idtj j jjd|jd|d|jd|d|d|d |d |jd |jd |jd |jd|jd|j d|j"d|j$d|j&d|j(d|j*d|j,d|j.d|j0d|j2d|j4d|j6d|j8d|j:d|j<d|j>} | j\jUd'd1} tLjNjQd2|d3| d4|jd5t_j`|| tc| j\|jdd67y#t@jB$rN} | jDdk(rbd | jFvrT|jH|js'tIjJd!|_ |js't||||||Yd} ~ y| jDd"k(rd#| jFvrtLjNjQd$|jRjUd%d&jUd'\} } tIjVd(| d)| d*d+d,d-d.d-/|_ n,#tHjX$rt[jXd0wxYwt||||||Yd} ~ y| d} ~ wwxYw)9z1Deploys the publisher model to a Vertex endpoint.Nr<locationr accept_eularKrrendpoint_display_namerspotreservation_affinityuse_dedicated_endpointenable_fast_tryoutrcontainer_commandrrcontainer_portscontainer_grpc_portscontainer_predict_routecontainer_health_route$container_deployment_timeout_secondscontainer_shared_memory_size_mbcontainer_startup_probe_exec&container_startup_probe_period_seconds'container_startup_probe_timeout_secondscontainer_health_probe_exec%container_health_probe_period_seconds&container_health_probe_timeout_secondsiz)provide a valid Hugging Face access tokenz2Please enter your Hugging Face read access token: iEULAzIThe End User License Agreement (EULA) of the model has not been accepted.@rrzThe model can be deployed only if the EULA of the model has been accepted. You can view it at https://console.cloud.google.com/vertex-ai/publishers/z/model-garden/z):zDo you want to accept the EULA?FTzEULA is not accepted.)message prompt_stringdefault cancel_on_no cancel_stringthrow_if_unattendedz6Please accept the EULA using the `--accept-eula` flag.rrrrz>` to find the status of the deployment long-running operation i6)r max_wait_ms)3rrrDeployrr:r;r<r=rJr rr r rrrrrrrrrrrrrrrrrrapitools_exceptions HttpErrorrcontentrPromptPasswordrrrrrPromptContinuerrrrrrr)rrrrr mg_clientrrKr deploy_ope publisherrrs r@r'r's^!--l%55&77l   !!&&..88:$$  *  , ",#'"@"@YY"66 $:: 22!4400 **!" 22#$,,%&"66'(!% < <)* $::+,.2-V-V-.)-(L(L/0&*%F%F1204/Z/Z34150\0\56%)$D$D78/3.X.X9:04/Z/Z;In%%c*2.,**!?#. 4;;-84 4  Y^^$$$ C  & &6  7199 D  * * 2..)3)B)B @* &..        # &AII"5 jj 8!JJ,,S1!4::3?i %44JKTTbckbllnp <1 $     ## D           gm6sDF*H55N AN%N:A,N''MN)M88NNNcDtjj|dS)zParse operation resource to the operation reference object. Args: operation_name: The operation resource to wait on Returns: The operation reference object z(aiplatform.projects.locations.operations)r7)rr8ParseRelativeName)operation_names r@rrYs(    - -; . rB)r4)1__doc__ __future__rrrr[apitools.base.pyrrr(googlecloudsdk.api_lib.air googlecloudsdk.api_lib.ai.modelsr r!googlecloudsdk.api_lib.monitoringr googlecloudsdk.api_lib.quotasr googlecloudsdk.command_lib.air r rgooglecloudsdk.corerrrrrgooglecloudsdk.core.consolerr|rErjrhrArNrurrrrrrrrrr'rr&rBr@r?s4&'%>0D44859=#*()2ED>FHHHKGC %!=<6>@@@D?4 )%:   #NDF DF$ +  <~)X')TH7V3lsl rB