Google Cloud Platform TLDR
— 4 min read
Below is a giant mess of some notes I've taken while studying for the Associate Cloud Engineer exam for the Google Cloud Platform. Just going to leave it archived here in case I want to use this as a pick up point when/if I prepare for the next certification tier.
SUPER excellent overall summary of all GCP services: About the GCP services | Overview | Google Cloud
Miscellaneous
Binary logging for point in time MySQL recoveries
Billing is either monthly or when you hit the threshhold
Use Service Accounts to provision access to applications!
-d for duration, in time — gsutil signurl -d 10m
~Cloud Dataflow.~ A fully managed service for transforming and enriching data in stream (real time) and batch (historical) modes with equal reliability and expressiveness
~Cloud Pub/Sub.~ As well as performing ingestion, Cloud Pub/Sub can also act as the glue between the loosely coupled systems.
~BigQuery.~ Place any data that you want to process or access later using a SQL interface into BigQuery.
~Cloud Bigtable.~ Place any data that you want to use for low-latency storage, or where you might want to get at a very small subset of a larger dataset quickly (key lookups as well as range scans), in Cloud Bigtable. Google’s sparsely populated NoSQL database which can scale to billions of rows, thousands of columns, and petabytes of data.
Networking
~Remote Desktop Protocol~ (RDP) is a proprietary protocol developed by Microsoft, which provides a user with a graphical interface to connect to another computer over a network connection. The user employs RDP client software for this purpose, while the other computer must run RDP server software. By default, the server listens on TCP port 3389 and UDP port 3389.
~Bastion hosts~ provide an external facing point of entry into a network containing private network instances. This host can provide a single point of fortification or audit and can be started and stopped to enable or disable inbound SSH communication from the Internet.
Enable ~flow logs~ to show and audit all network traffic inside of the VPC
Instances
Managed group instance allows auto scaling … auto scaling allows detecting and replacing failing instances, also has auto-healing, multi-zone deployments, and automatic updating
Unmanaged instance groups allow you control to manage the instances yourself, not desireable
An instance template is a resource that you can use to create VM instances and managed instance groups. Instance templates define the machine type, boot disk image or container image, labels, and other instance properties. You can then use an instance template to create a managed instance group or to create individual VM instances.
A ~preemptible~ VM is an instance that you can create and run at a much lower price than normal instances. However, Compute Engine might terminate (preempt) these instances if it requires access to those resources for other tasks. ~Preemptible~ instances are excess Compute Engine capacity, so their availability varies with usage.
~min-idle-instances~ property can be set to have minimum idle instances which would be always running.
gcloud compute instances add-metadata -> specific instance scope
gcloud compute project-info add-metadata -> entire project scope
Setup & Initialization & Deploy
App engine regions cannot be changed once set
Standard practice is to enable/disable API Libraries from the GCP Console
Cloud shell provisions 5gb of persistent free storage mounted at \$HOME ~ for the purpose of saving things such as your .bashrc config
Deployment Manager allows you to specify all the resources needed for your application in a declarative format using yaml. Allow reuse of common deployment paradigms such as a load balanced, auto-scaled instance group.
Deployment Manager - use startup-script metadata key to launch application quickly without complexity
gcloud config configurations create [NAME] -> creates named config
gcloud config configurations activate [NAME] -> activate named config
gcloud config configurations list -> lists named configs
Kubernetes
Use gcloud to create/manage/resize a kubernetes cluster. Once the cluster is created you can use kubectl to manage the object deployments.
You can use kubectl set to make changes to an object’s image, resources (compute resource such as CPU and memory), or selector fields. Set will perform a rolling update with minimal downtime.
kubectl get deployments -> list kubernetes deployments … deployments represent multiple identical Pods with no unique identities, replicas.
You can use kubectl apply to apply a new configuration file to an existing controller object. kubectl apply is useful for making multiple changes to a resource, through configuration files.
Autoscaling is configured at the engine level, not instance group!
Use service of type ClusterIP to communicate between deployments
File Transferring
Multi-threaded/processed: Useful when transferring large number of files.
Parallel composite uploads: Splits large files, transfers chunks in parallel, and composes at destination.
Security & Auth
Firewalls/Resources in GCP can utilize ‘network-tags’ to allow more granular controls
Kubernetes Secrets are secure objects which store sensitive data, such as passwords, OAuth tokens, and SSH keys, in your Kubernetes clusters. Storing sensitive data in Secrets is more secure than plaintext ConfigMaps or in Pod specifications.
gcloud iam roles copy — copy IAM roles onto another project
Computing
Ordered by level of abstraction:
- Cloud Functions (Serverless, Event Driven)
- App Engine (Partially managed by Google)
- Compute Engine (Raw instances of VMs, self managed, cheaper)
- Your home server or physical server rack (Bare metal, completely self managed)
gcloud compute ssh -> easiest and quickest way to use ssh with a compute engine instance
App Engine
App Engine Standard: Scales very rapidly from zero to load, more strict environment rules, can potentially have very low costs
App Engine Flexible: SSH capability, for more ‘stable’ applications, gradual scaling up and down, run with Docker containers
App Engine can split/route 100% traffic to previous versions easily in GCP Console if necessary
Deploy App Engine: gcloud app deploy app.yaml
Storage
- Multi-regional: Frequently accessed around the world
- Regional: Frequently accessed data in same region as your DataProc/Compute Engine (analytics)
- Nearline Storage: Data store for monthly access
- Coldline Storage: Data store for yearly access / disaster recovery / archive
Use cases: Mobile Sdks? Cloud Storage Firebase Non-Structured Data? Cloud Storage(Buckets, Global, Instant) Structured Relational? Cloud SQL(Small Data) -> Cloud Spanner Horizontal Structured Non-Relational? Cloud Firestore Analytics? BigQuery(SQL) -> BigTable (Updates/Fast/Large/NoSQL) Redis? Cloud Memorystore
Failover/Read storage replicas must be in the same region!