\BdZddlmZddlmZddlmZddlZddlmZddl m Z ddl m Z dd lmZdd lmZdd lmZddlmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZej:ej<j>ej<j@ej<jBejDGddejFZ$y)zModel Garden deploy command.)absolute_import)division)unicode_literalsN) exceptions) operations)client)apis) arg_parsers)base) constants) endpoint_util)flags)model_garden_utils) region_util) validation) propertiesc&eZdZdZedZdZy)DeployacDeploy a model in Model Garden to a Vertex AI endpoint. ## EXAMPLES To deploy a Model Garden model `google/gemma2/gemma2-9b` under project `example` in region `us-central1`, run: $ gcloud ai model-garden models deploy --model=google/gemma2@gemma-2-9b --project=example --region=us-central1 To deploy a Hugging Face model `meta-llama/Meta-Llama-3-8B` under project `example` in region `us-central1`, run: $ gcloud ai model-garden models deploy --model=meta-llama/Meta-Llama-3-8B --hugging-face-access-token={hf_token} --project=example --region=us-central1 c ( tjdddj|tjdddj|tjddd j|tj|d t j tjd d dj|tjdddj|tjddtdj|tjdddddj|tjdddddj|tjdtjtttjddgdj|tjd dddd!"j|tjd#dddd$"j|tjd%dddd&"j|tjd'd()j||jd*d+tjtjd,-|jd.tjd/tjd01|jd2d3tjd45|jd6d7tjtjd8d9:tjd;-|jdd?)|jd@dA)|jdBtdC|jdDtdE|jdFtjdGdHI|jdJtdK|jdLtdM|jdNtjdOdPI|jdQtdR|jdStdTy)UN--modelTaThe model to be deployed. If it is a Model Garden model, it should be in the format of `{publisher_name}/{model_name}@{model_version_name}, e.g. `google/gemma2@gemma-2-2b`. If it is a Hugging Face model, it should be in the convention of Hugging Face models, e.g. `meta-llama/Meta-Llama-3-8B`. If it is a Custom Weights model, it should be in the format of `gs://{gcs_bucket_uri}`, e.g. `gs://-model-garden-public-us/llama3.1/Meta-Llama-3.1-8B-Instruct`.)requiredhelpz--hugging-face-access-tokenFzThe access token from Hugging Face needed to read the model artifacts of gated models. It is only needed when the Hugging Face model to deploy is gated.z--endpoint-display-namez5Display name of the endpoint with the deployed model.zto deploy the model) prompt_funcz--machine-typezThe machine type to deploy the model to. It should be a supported machine type from the deployment configurations of the model. Use `gcloud ai model-garden models list-deployment-config` to check the supported machine types.)rrz--accelerator-typezThe accelerator type to serve the model. It should be a supported accelerator type from the verified deployment configurations of the model. Use `gcloud ai model-garden models list-deployment-config` to check the supported accelerator types.z--accelerator-countzSThe accelerator count to serve the model. Accelerator count should be non-negative.)rtyperz --accept-eulazNWhen set, the user accepts the End User License Agreement (EULA) of the model. store_true)ractiondefaultrz--asynchronouszaIf set to true, the command will terminate immediately and not keep polling the operation status.z--reservation-affinity)reservation-affinity-typekeyvaluesr)spec required_keyszA ReservationAffinity can be used to configure a Vertex AI resource (e.g., a DeployedModel) to draw its Compute Engine resources from a Shared Reservation, or exclusively from on-demand capacity.)rrz--spotz5If true, schedule the deployment workload on Spot VM.)rrrrz--use-dedicated-endpointzIf true, the endpoint will be exposed through a dedicated DNS. Your request to the dedicated DNS will be isolated from other users' traffic and will have better performance and reliability.z--enable-fast-tryoutzIf True, model will be deployed using faster deployment path. Useful for quick experiments. Not for production workloads. Only available for most popular models with certain machine types.z--container-image-uriz{ URI of the Model serving container file in the Container Registry (e.g. gcr.io/myproject/server:latest). )rz--container-env-varsz KEY=VALUEz8List of key-value pairs to set as environment variables.)metavarrrrz--container-commandCOMMANDzm Entrypoint for the container image. If not specified, the container image's default entrypoint is run. )rr#rrz--container-argsARGz Comma-separated arguments passed to the command run by the container image. If not specified and no `--command` is provided, the container image's default command is used. )r#rrz--container-portsPORTi) element_typezd Container ports to receive http requests at. Must be a number between 1 and 65535, inclusive. z--container-grpc-portszd Container ports to receive grpc requests at. Must be a number between 1 and 65535, inclusive. z--container-predict-routez>HTTP path to send prediction requests to inside the container.z--container-health-routez8HTTP path to send health checks to inside the container.z&--container-deployment-timeout-secondszDeployment timeout in seconds.z!--container-shared-memory-size-mbz` The amount of the VM memory to reserve as the shared memory for the model in megabytes. z--container-startup-probe-execSTARTUP_PROBE_EXECz Exec specifies the action to take. Used by startup probe. An example of this argument would be ["cat", "/tmp/healthy"]. )rr#rz(--container-startup-probe-period-secondszh How often (in seconds) to perform the startup probe. Default to 10 seconds. Minimum value is 1. z)--container-startup-probe-timeout-secondszm Number of seconds after which the startup probe times out. Defaults to 1 second. Minimum value is 1. z--container-health-probe-execHEALTH_PROBE_EXECz Exec specifies the action to take. Used by health probe. An example of this argument would be ["cat", "/tmp/healthy"]. z'--container-health-probe-period-secondszg How often (in seconds) to perform the health probe. Default to 10 seconds. Minimum value is 1. z(--container-health-probe-timeout-secondszl Number of seconds after which the health probe times out. Defaults to 1 second. Minimum value is 1. )r Argument AddToParserrAddRegionResourceArgrPromptForOpRegionintr ArgDictstrArgList add_argument UpdateAction BoundedInt)parsers ,lib/surface/ai/model_garden/models/deploy.pyArgsz Deploy.ArgsCsMM L k&MM% : k&MM! Dk& %;3P3P MM ,  k&MM Q  k&MM 'k&MM  k&MM 2 k&MM  -0%--/ 77   # k&MM D  k&MM" P k&MM M k&MM  k&   "'' G    "''     "     k.D.DQ.N O''      k.D.DQ.N O''   # M " G 0  -  +  (  "$  2  3  '  "#  1  2 c | |jjd}|stj|tj|j |j jj}|jd|_tj}d|jv}tj|d5|r t|jt|j cxk(rt|j"k(snt%j&ddd}|jrt)j*||j|j |j"t-j.tj0tj2|}|j4j7|j|j4j6j9|j |j" }tj||j5d j;d t=t?j>jAd d dg}tCjD} tGjH|} |j r |j n|} t)jJ||| |j| | dddnLtCjD} |rc|jjMjAd\} } | jOd| d| d}d j;| | ddg}| d| }nr|jjMjAd\} } | jOd| d|}d j;| |jAdddg}d| d|}t)jV||}t)j*||jXjZj\t=|jXjZj^|jXjZj`tbjdjfjhjkdtj||j5tCjD} tGjH|} |j r |j n|} t)jJ||jXjZ| || | ddddddy#1swYxYw#tPjR$r$t%jTd|jdwxYw#tPjR$r$t%jTd|jdwxYw#1swYxYw#1swYyxYw)Nzgs:// locationsId@z us-central1)regionz:--machine-type, --accelerator-type and --accelerator-countz Arguments for MachineType, AcceleratorType and AcceleratorCount must either all be provided or all be empty for custom weights model deployment.) machine_typeaccelerator_typeaccelerator_count) machineTypeacceleratorTypeacceleratorCount-zcustom-weights.rz mg-cli-deploy)version/z publishers/z/models/T) model_nameis_hugging_face_modelrzF is not a supported Hugging Face model for deployment in Model Garden.hfzF is not a supported Model Garden model for deployment in Model Garden.r')6model startswithrValidateModelGardenModelArgsValidateDisplayNameendpoint_display_nameCONCEPTSr=ParseAsDictr BETA_VERSIONr AiplatformEndpointOverridesboolr>r?r@ c_exceptionsInvalidArgumentExceptionrCheckAcceleratorQuotar GetClientInstanceAI_PLATFORM_API_NAMEAI_PLATFORM_API_VERSIONMESSAGES_MODULE'GoogleCloudAiplatformV1beta1MachineSpecAcceleratorTypeValueValuesEnumjoinr1timesplit client_mgModelGardenClientrOperationsClientrlowerGetPublisherModelapitools_exceptionsHttpNotFoundErrorUnknownArgumentExceptionGetDeployConfigdedicatedResources machineSpecrArBrCrVALUESapi_endpoint_overrides aiplatformSet)selfargsis_custom_weights_model region_refrF is_hf_model machine_specrdefault_endpoint_name mg_clientoperation_client endpoint_namepublisher_namerHpublisher_model api_model_argmodel_and_version_name deploy_configs r7Runz Deploy.RunGs`"jj33G< "--d3""4#=#=>%%++-J##%m4DK$$GTZZ'K  2 2   ! "" #D))* ,D**+ ,55J#       2 2,,#44 $ 6 6  )),,//8 &  //WW++$44\\{{'' $55 X , 6 6 DKK #&(($))+$$S)!,, #   113)'88I ++(((   # #jj  !  8//1 '+zz'7'7'9'?'?'D $.* '99((8 M&*:O#&((z4A# ,,Aj\:-48::3C3C3E3K3KC3P 0.0 '99n-X6L5MNO#&(($**3/2, #  N+84J3KL +:: / 00 &99EEQQ 00<<LL,>>JJ[[   00;;??E 6 6 DKK  113)'88I ++(((   # #..::   G  R  L%66 77::,99  &%66 77::,//  B  G  srEV2B*T&A V2T2-A V27U,DV2"A3V&V2&T/ +V227U))V2,7V##V2&V/ +V22V;N)__name__ __module__ __qualname____doc__ staticmethodr8rr9r7rr's$ .AAF_ r9r)%r __future__rrrr`apitools.base.pyrrggooglecloudsdk.api_lib.air&googlecloudsdk.api_lib.ai.model_gardenrrbgooglecloudsdk.api_lib.utilr googlecloudsdk.callioper r rVgooglecloudsdk.command_lib.air r rrrrgooglecloudsdk.corer ReleaseTracks ReleaseTrackALPHABETAGADefaultUniverseOnlyCommandrrr9r7rs#&' >0F,/(>37/<54*T..33T5F5F5I5I{ T\\{ { r9