I^dZddlmZddlmZddlmZdZGddejZ Gdd ejZ Gd d ejZ Gd d ejZ GddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGddejZGd d!ejZGd"d#ejZGd$d%ejZGd&d'ejZGd(d)ejZGd*d+ejZGd,d-ejZGd.d/ejZGd0d1ejZGd2d3ejZ Gd4d5ejZ!Gd6d7ejZ"ejFe d8d9ejHe jJd:d;ejHe jJd)?zNGenerated message classes for gkerecommender version v1. GKE Recommender API )absolute_import)messages)encodinggkerecommenderceZdZdZej dej jZej dZ y)AmountakRepresents an amount of money in a specific currency. Fields: nanos: Output only. Number of nano (10^-9) units of the amount. The value must be between -999,999,999 and +999,999,999 inclusive. If `units` is positive, `nanos` must be positive or zero. If `units` is zero, `nanos` can be positive, zero, or negative. If `units` is negative, `nanos` must be negative or zero. For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000. units: Output only. The whole units of the amount. For example if `currencyCode` is `"USD"`, then 1 unit is one US dollar. variantN) __name__ __module__ __qualname____doc__ _messages IntegerFieldVariantINT32nanosunitsYlib/googlecloudsdk/generated_clients/apis/gkerecommender/v1/gkerecommender_v1_messages.pyrrs;  !) I,=,=,C,C D% )  #%rrceZdZdZej ddZej ddZejdejjZ ejdZ y) CostaCost for running a model deployment on a given instance type. Currently, only USD currency code is supported. Fields: costPerMillionInputTokens: Optional. The cost per million input tokens. $/input token = ($/output token) / output-to-input-cost-ratio. costPerMillionOutputTokens: Optional. The cost per million output tokens, calculated as: $/output token = GPU $/s / (1/output-to-input-cost-ratio * input tokens/s + output tokens/s) outputInputCostRatio: Optional. The output-to-input cost ratio. This determines how the total GPU cost is split between input and output tokens. If not provided, `4.0` is used, assuming a 4:1 output:input cost ratio. pricingModel: Optional. The pricing model used to calculate the cost. Can be one of: `3-years-cud`, `1-year-cud`, `on-demand`, `spot`. If not provided, `spot` will be used. rr r r N)r rrrr MessageFieldcostPerMillionInputTokenscostPerMillionOutputTokens FloatFieldrFLOAToutputInputCostRatio StringField pricingModelrrrrr"sf$5i44XqA5y55hB---a9J9J9P9PQ&&&q),rrc~eZdZdZej dZejddZej dZ y)FetchBenchmarkingDataRequestaRequest message for GkeInferenceQuickstart.FetchBenchmarkingData. Fields: instanceType: Optional. The instance type to filter benchmarking data. Instance types are in the format `a2-highgpu-1g`. If not provided, all instance types for the given profile's `model_server_info` will be returned. Use GkeInferenceQuickstart.FetchProfiles to find available instance types. modelServerInfo: Required. The model server configuration to get benchmarking data for. Use GkeInferenceQuickstart.FetchProfiles to find valid configurations. pricingModel: Optional. The pricing model to use for the benchmarking data. Defaults to `spot`. r ModelServerInfor rN) r rrrrr$ instanceTypermodelServerInfor%rrrr'r';sB '&&q),*I**+B "I " "9a$ ?'rrGceZdZdZej dZej dZejddZ ejddZ ejdd Z y ) GenerateOptimizedManifestRequestafRequest message for GkeInferenceQuickstart.GenerateOptimizedManifest. Fields: acceleratorType: Required. The accelerator type. Use GkeInferenceQuickstart.FetchProfiles to find valid accelerators for a given `model_server_info`. kubernetesNamespace: Optional. The kubernetes namespace to deploy the manifests in. modelServerInfo: Required. The model server configuration to generate the manifest for. Use GkeInferenceQuickstart.FetchProfiles to find valid configurations. performanceRequirements: Optional. The performance requirements to use for generating Horizontal Pod Autoscaler (HPA) resources. If provided, the manifest includes HPA resources to adjust the model server replica count to maintain the specified targets (e.g., NTPOT, TTFT) at a P50 latency. Cost targets are not currently supported for HPA generation. If the specified targets are not achievable, the HPA manifest will not be generated. storageConfig: Optional. The storage configuration for the model. If not provided, the model is loaded from Huggingface. r r r(rr>r StorageConfigr=N) r rrrrr$acceleratorTypekubernetesNamespacerr*rE storageConfigrrrrLrLsm,*I))!,/- --a0*I**+aPerformance requirements for a profile and or model deployment. Fields: targetCost: Optional. The target cost for running a profile's model server. If not provided, this requirement will not be enforced. targetNtpotMilliseconds: Optional. The target Normalized Time Per Output Token (NTPOT) in milliseconds. NTPOT is calculated as `request_latency / total_output_tokens`. If not provided, this target will not be enforced. targetTtftMilliseconds: Optional. The target Time To First Token (TTFT) in milliseconds. TTFT is the time it takes to generate the first token for a request. If not provided, this target will not be enforced. rr r r rN) r rrrrr targetCostrrrtargetNtpotMillisecondstargetTtftMillisecondsrrrr>r>s_ &y%%fa0*2I221i>O>O>U>UV1911!Y=N=N=T=TUrr>czeZdZdZej dddZejdejjZ ejdejjZ ejd ejjZejd ejjZy ) PerformanceStatsaPerformance statistics for a model deployment. Fields: cost: Output only. The cost of running the model deployment. ntpotMilliseconds: Output only. The Normalized Time Per Output Token (NTPOT) in milliseconds. This is the request latency normalized by the number of output tokens, measured as request_latency / total_output_tokens. outputTokensPerSecond: Output only. The number of output tokens per second. This is the throughput measured as total_output_tokens_generated_by_server / elapsed_time_in_seconds. queriesPerSecond: Output only. The number of queries per second. Note: This metric can vary widely based on context length and may not be a reliable measure of LLM throughput. ttftMilliseconds: Output only. The Time To First Token (TTFT) in milliseconds. This is the time it takes to generate the first token for a request. rr Tr.r r rrr=N)r rrrrrcostrrrntpotMillisecondsoutputTokensPerSecondr!r"queriesPerSecondttftMillisecondsrrrrqrqs&   D 9$,i,,Q 8I8I8O8OP0)00IDQ()((!<-% %%a(+rr-c`eZdZdZej dej jZy)rxzResources used by a model deployment. Fields: acceleratorCount: Output only. The number of accelerators (e.g., GPUs or TPUs) used by the model deployment on the Kubernetes node. r r N) r rrrrrrracceleratorCountrrrrxrxs*,Y++Ay7H7H7N7NOrrxc eZdZdZGddej ZGddej ZejddZ ejdZ ejddd Z ejd Z ejd Zejd ZejdZej"dd ZejdZejdZejdZejdZy)StandardQueryParametersa4Query parameters accepted by all methods. Enums: FXgafvValueValuesEnum: V1 error format. AltValueValuesEnum: Data format for response. Fields: f__xgafv: V1 error format. access_token: OAuth access token. alt: Data format for response. callback: JSONP fields: Selector specifying which fields to include in a partial response. key: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token. oauth_token: OAuth 2.0 token for the current user. prettyPrint: Returns response with indentations and line breaks. quotaUser: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters. trace: A tracing token of the form "token:" to include in api requests. uploadType: Legacy upload protocol for media (e.g. "media", "multipart"). upload_protocol: Upload protocol for media (e.g. "raw", "multipart"). ceZdZdZdZdZdZy)*StandardQueryParameters.AltValueValuesEnumzData format for response. Values: json: Responses with Content-Type of application/json media: Media download with context-dependent Content-Type proto: Responses with Content-Type of application/x-protobuf rr r N)r rrrjsonmediaprotorrrAltValueValuesEnumrs D E ErrceZdZdZdZdZy)-StandardQueryParameters.FXgafvValueValuesEnumzVV1 error format. Values: _1: v1 error format _2: v2 error format rr N)r rrr_1_2rrrFXgafvValueValuesEnumrs B Brrr r rr)defaultrr=r?T N)r rrrrEnumrr EnumFieldf__xgafvr$ access_tokenaltcallbackfieldskey oauth_token BooleanField prettyPrint quotaUsertrace uploadTypeupload_protocolrrrrrs 4 9>>  inn !Y !8! <(&&&q), 0!VD# "Y " "1 %( 9  #& a #% %%a(+& &&q$7+#i##A&) )   #%$y$$R(*)I))"-/rrcXeZdZdZej dZej dZy)rMaStorage configuration for a model deployment. Fields: modelBucketUri: Optional. The Google Cloud Storage bucket URI to load the model from. This URI must point to the directory containing the model's config file (`config.json`) and model weights. A tuned GCSFuse setup can improve LLM Pod startup time by more than 7x. Expected format: `gs:///`. xlaCacheBucketUri: Optional. The URI for the GCS bucket containing the XLA compilation cache. If using TPUs, the XLA cache will be written to the same path as `model_bucket_uri`. This can speed up vLLM model preparation for repeated deployments. r r N)r rrrrr$modelBucketUrixlaCacheBucketUrirrrrMrMs. )9((+.+i++A.rrMceZdZdZej dej jZej dej jZ y)rhzRepresents a range of throughput values in tokens per second. Fields: max: Output only. The maximum value of the range. min: Output only. The minimum value of the range. r r r Nrbrrrrhrh(rerrhrz$.xgafvr1r2N)&r __future__rapitools.base.protorpcliterrapitools.base.pyrpackageMessagerrr'r,r2r6r9r<rGrLrRrWrYr[rSrar(rHr>rqr-rxrrMrhAddCustomJsonFieldMappingAddCustomJsonEnumMappingrrrrrsC '<% $Y  $$*9  *2*9#4#4**@I$5$5@ +y'8'8 + + 1 1 + +)++ + $Q9,,$QN@I--@*=y'8'8=<- (9(9-('I4E4E':'Y->->'0'y'8'8'( "** " Cy(( C0i''0*!!!114>r