Although the simple ‘gcloud container operations list‘ command is the easiest way to find recent upgrade events on your GKE cluster or nodepool, it returns only the recent events and does not provide a historical record.
If you need to look at historical events, you can use Logs Explorer web UI or use the ‘gcloud logging read’ command from the CLI.
Querying Logs Explorer from CLI
The ‘gcloud logging read‘ command has a ‘freshness‘ flag that determines how far back you want to query. It can be specified in hours (e.g. 3h), days (e.g. 90d), or years (e.g. 1y).
If not explicitly set, the default is one day. So if you are looking for historical events, be sure to set this flag appropriately.
Master control plane upgrade
From the Logs Explorer web UI, select a time frame and use the following search criteria:
resource.type="gke_cluster"
log_id("cloudaudit.googleapis.com/activity")
protoPayload.methodName="google.container.internal.ClusterManagerInternal.UpdateClusterInternal"
operation.last=true
Open any resulting events and look at the ‘protoPayload.metadata’ field for the old and new version.
This can be queried at the gcloud CLI using this equivalent command:
gcloud logging read 'resource.type="gke_cluster" AND log_id("cloudaudit.googleapis.com/activity") AND protoPayload.methodName="google.container.internal.ClusterManagerInternal.UpdateClusterInternal" AND operation.last=true' --freshness=1d --limit=10 --format="json(protoPayload.metadata)"
Nodepool upgrade
From the Logs Explorer web UI, select a time frame and use the following search criteria:
resource.type="gke_nodepool"
log_id("cloudaudit.googleapis.com/activity")
protoPayload.methodName="google.container.v1.ClusterManager.UpdateNodePool"
operation.first=true
Open any resulting events and look at the ‘protoPayload.request.nodeVersion’ field for the new version.
This can be queried at the gcloud CLI using this equivalent command:
gcloud logging read 'resource.type="gke_nodepool" AND log_id("cloudaudit.googleapis.com/activity") AND protoPayload.methodName="google.container.v1.ClusterManager.UpdateNodePool" AND operation.first=true' --freshness=1d --limit=10 --format="json(protoPayload.request.nodeVersion)"
Nodepool scaling
resource.type="gke_nodepool"
log_id("cloudaudit.googleapis.com/activity")
protoPayload.methodName="google.container.v1.ClusterManager.SetNodePoolSize"
operation.first=true
Open any resulting events and look at the ‘protoPayload.request.nodeCount’ field for the new size request.
This can be queried at the gcloud CLI using this equivalent command:
gcloud logging read 'resource.type="gke_nodepool" AND log_id("cloudaudit.googleapis.com/activity") AND protoPayload.methodName="google.container.v1.ClusterManager.SetNodePoolSize" AND operation.first=true' --freshness=1d --limit=10 --format="json(protoPayload.request.nodeCount)"
REFERENCES
Google doc, querying audit logs
Google doc, how-to for audit logging
stackoverflow, how to find historical version info after GKE cluster upgrade
gkesecurity.guide – list of different resource audit types
command ‘gcloud container operations list’
Google doc, migrating from activity log to audit logs and example queries