Redis vs DynamoDB
Redis Sentinel VS Cluster
Redis Sentinel runs as a separate program. You should have atleast 3 Sentinel instances monitoring a master instance and its slaves. Sentinel instances try to find consensus when doing a failover and only an odd number of instances will prevent most problems, 3 being the minimum. In this case one of the Sentinel instances can go down and a failover will still work as (hopefully) the other two instances reach consensus which slave to promote.
Redis Cluster is a data sharding solution with automatic management, handling failover and replication.
K8s – SecurityContext -Notes /Openshift SCC
apiVersion: v1
kind: Pod
metadata:
name: security-context-demo
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
volumes:
- name: sec-ctx-vol
emptyDir: {}
containers:
- name: sec-ctx-demo
image: busybox:1.28
command: [ "sh", "-c", "sleep 1h" ]
volumeMounts:
- name: sec-ctx-vol
mountPath: /data/demo
securityContext:
allowPrivilegeEscalation: false
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
In the configuration file, the runAsUser field specifies that for any Containers in the Pod, all processes run with user ID 1000. The runAsGroup field specifies the primary group ID of 3000 for all processes within any containers of the Pod. If this field is omitted, the primary group ID of the containers will be root(0). Any files created will also be owned by user 1000 and group 3000 when runAsGroup is specified. Since fsGroup field is specified, all processes of the container are also part of the supplementary group ID 2000. The owner for volume /data/demo and any files created in that volume will be Group ID 2000.
Openshift UID: https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids
UID Explanations:
https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids
https://andreaskaris.github.io/blog/openshift/scc/
https://www.redhat.com/sysadmin/security-context-constraint-configuration
oc get project projectname -o yaml -it will output the
During the creation of a project or namespace, OpenShift assigns a User ID (UID) range, a supplemental group ID (GID) range, and unique SELinux MCS labels to the project or namespace. By default, no range is explicitly defined for fsGroup, instead, by default, fsGroup is equal to the minimum value of the “openshift.io/sa.scc.supplemental-groups” annotation
The range of UIDs, GIDs, and SELinux MCS labels are unique to the project and there will not be overlap with the UIDs or GIDs assigned to other projects.
When a Pod is deployed into the namespace, by default, OpenShift will use the first UID and first GID from this range to run the Pod. Any attempt by a Pod definition to specify a UID outside the assigned range will fail and requires special privileges.
SELinu: Security Enhanced Linux
# Creating ServiceAccount with “anyuid” SCC $ oc create sa sa-with-anyuid $ oc adm policy add-scc-to-user anyuid -z sa-with-anyuid
Service account is like the user,. and scc is like the role (anyuid is the scc)
oc get scc| grep anyuid
oc describe scc anyuid
7 scc are added to the cluster by default, and only cluster admin could see it.
In OpenShift the Security Context Constraints (SCC) are used to manage and control the permissions and capabilities granted to a Pod. There are eight (8) SCC pre-defined in an OpenShift 4.4 cluster and, by default, each namespace is created with three (3) ServiceAccounts.
oc get scc
oc get -o yaml –export scc/privileged
Lessons learned: we can apply multiple SCCs to a ServiceAccount. The default SCC is always restricted. The matching goes by priority first, then most restrictive policy, then by name. And if a policy does not match a specific request, another one might “jump in” and “help out”.
https://andreaskaris.github.io/blog/openshift/scc/ –the prority
Highest priority first, nil is considered a 0 priority
If priorities are equal, the SCCs will be sorted from most restrictive to least restrictive
If both priorities and restrictions are equal the SCCs will be sorted by name
K8s(Volumes/PVC/PV)
AWS EBS create the storageClass
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
type: io1
iopsPerGB: "10"
fsType: ext4
https://kubernetes.io/docs/concepts/storage/storage-classes/
provisioner : ebs , azure files systems
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: example-nfs
provisioner: example.com/external-nfs
parameters:
server: nfs-server.example.com
path: /share
readOnly: "false"
Create dynamic provision PVC :
Users request dynamically provisioned storage by including a storage class in their PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: claim1
spec:
accessModes:
- ReadWriteOnce
storageClassName: fast
resources:
requests:
storage: 30Gi
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 8Gi
storageClassName: slow
selector:
matchLabels:
release: "stable"
matchExpressions:
- {key: environment, operator: In, values: [dev]}
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: myfrontend
image: nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myclaim
PVs are resources in the cluster. PVCs are requests for those resources and also act as claim checks to the resource
Create PV:
apiVersion: v1
kind: PersistentVolume
metadata:
name: task-pv-volume
labels:
type: local
spec:
storageClassName: manual
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: "/mnt/data"
Create PVC:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: task-pv-claim
spec:
storageClassName: manual
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 3Gi
After you create the PersistentVolumeClaim, the Kubernetes control plane looks for a PersistentVolume that satisfies the claim’s requirements. If the control plane finds a suitable PersistentVolume with the same StorageClass, it binds the claim to the volume.
Create NFS no need to create the storage class….
apiVersion: v1
kind: PersistentVolume
metadata:
name: <nfs pv name>
spec:
capacity:
storage:10g
accessModes:
– ReadWriteOnce
nfs:
path: <nfs share path>
server: <nfs server ip or hostname>
so are the openshift(pv 7 pvc :
A PV can have a class, which is specified by setting the storageClassName attribute to the name of a StorageClass. A PV of a particular class can only be bound to PVCs requesting that class. A PV with no storageClassName has no class and can only be bound to PVCs that request no particular class.
When a PVC specifies a selector in addition to requesting a StorageClass, the requirements are ANDed together: only a PV of the requested class and with the requested labels may be bound to the PVC.
Claim as volumes
LimitRange/Volumes
apiVersion: v1
kind: LimitRange
metadata:
name: mem-min-max-demo-lr
spec:
limits:
- max:
memory: 1Gi
min:
memory: 500Mi
type: Container
apiVersion: v1
kind: LimitRange
metadata:
name: cpu-min-max-demo-lr
spec:
limits:
- max:
cpu: "800m"
min:
cpu: "200m"
type: Container
The Container is running in a namespace that has a default memory limit, and the Container is automatically assigned the default limit. Cluster administrators can use a LimitRange to specify a default value for the memory limit.
Volume
1.emptyDir : this Pod has a Volume of type emptyDir that lasts for the life of the Pod, even if the Container terminates and restarts. Here is the configuration file for the Pod:
Note: A container crashing does not remove a Pod from a node. The data in an emptyDir volume is safe across container crashes.
oc get limitrange rangeName -o yaml
with the limitRange: when create the container without request, it will apply with the LimitRange
https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/
K8s -DNS- Headless Service-Access Services
DNS Records -https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
What objects get DNS records?
- Services
- Pods
Services:
“Normal” (not headless) Services are assigned DNS A and/or AAAA records, depending on the IP family or families of the Service, with a name of the form my-svc.my-namespace.svc.cluster-domain.example. This resolves to the cluster IP of the Service.
Headless Services (without a cluster IP) Services are also assigned DNS A and/or AAAA records, with a name of the form my-svc.my-namespace.svc.cluster-domain.example. Unlike normal Services, this resolves to the set of IPs of all of the Pods selected by the Service. Clients are expected to consume the set or else use standard round-robin selection from the set
Access Services Running on Cluster: https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster-services/
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app.kubernetes.io/name: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376
This specification creates a new Service object named “my-service”, which targets TCP port 9376 on any Pod with the app.kubernetes.io/name=MyApp label.
without selector, the endpointsslice of the services will nto be created auto, so have to create endpointslice – refer: https://kubernetes.io/docs/concepts/services-networking/service/
kubectl expose deployment source-ip-app –name=clusterip –port=80 –target-port=8080
kubectl expose deployment source-ip-app –name=nodeport –port=80 –target-port=8080 –type=NodePort
NodePort vs LoadBalancer:
k8s- command
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/imperative-command/
playground:https://killercoda.com/
kubectl create service clusterip my-svc –clusterip=”None” -o yaml –dry-run=client > srv.yaml –
kubectl create -f serv.yaml
kubectl delete -f serv.yaml
kubectl get -f <filename|url> -o yaml
kubectl get -h (get the help options for all options)
kubectl get -f <filename|url> -o yaml > test_delete.yaml
kubctl delete -f test_delete.yaml
kubectl apply -f applytest.yaml
kubectl create -f test.yaml
kubectl get test.yaml -o yaml
The last-applied-configuration annotation has been updated with the new image.
To-be-reviewed
How to use kustomize command: https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/
kubectl kustomize <kustomization_directory>
eg:
echo -n 'admin' > ./username.txt
echo -n '1f2d1e2e67df' > ./password.txt
cat <<EOF >./kustomization.yaml
secretGenerator:
- name: example-secret-1
files:
- password.txt
EOF
- mkdir test_dir
- cd test_dir
- inside the directory test_dir, and create two files:
cat <<EOF >application.properties
FOO=Bar
EOF
cat <<EOF >./kustomization.yaml
configMapGenerator:
- name: example-configmap-1
files:
- application.properties
EOF
cat <<EOF >./kustomization.yaml
secretGenerator:
- name: database-creds
files:
- username.txt
EOF
so under test_dir,there are two files created.
4.run: kubectl kustomize ./
5. kubectl apply -k ./
Patch : https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/
secrets : created.
kubectl describe secret secretname
kubectl get secrets
Secrets: Type of secrets and its usecase:
https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-files-from-a-pod
a. VolumeMount : secret -myScrets : if two have keys. and then will create two files under :/etc/foo
(Inside the container that mounts a secret volume, the secret keys appear as files. The secret values are base64 decoded and stored inside these files.)
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
volumes:
- name: foo
secret:
secretName: mysecret
defaultMode: 0400
so in order to make decisive of the MountPath
could be :
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecret
items:
- key: username
path: my-group/my-username
Get into a container that running in one POD
kubectl exec -i -t PODNAME — /bin/bash
kubectl exec envar-demo — printenv
Above command is to list the ENV in the POD
Use Pod fields as values for environment variables: https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/
env:
- name: MY_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: MY_POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: MY_POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: MY_POD_SERVICE_ACCOUNT
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
restartPolicy: Never
StatefulSet: https://kubernetes.io/docs/tasks/run-application/run-replicated-stateful-application/
Java Test-Mockito
Learn Kubernetes /Openshift -Notes
https://kubernetes.io/docs/tutorials/
https://kubernetes.io/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive/
Notes from learning :
Object Names and IDs
Each object in your cluster has a Name that is unique for that type of resource. Every Kubernetes object also has a UID that is unique across your whole cluster.
For example, you can only have one Pod named myapp-1234 within the same namespace, but you can have one Pod and one Deployment that are each named myapp-1234.
For non-unique user-provided attributes, Kubernetes provides labels and annotations
Labels vs Annotations:
Labels are key/value pairs that are attached to objects, such as pods. Labels are intended to be used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system. Labels can be used to organize and to select subsets of objects. Labels can be attached to objects at creation time and subsequently added and modified at any time. Each object can have a set of key/value labels defined. Each Key must be unique for a given object.
Labels allow for efficient queries and watches and are ideal for use in UIs and CLIs. Non-identifying information should be recorded using annotations
namespace:
Namespaces and DNS
When you create a Service, it creates a corresponding DNS entry. This entry is of the form <service-name>.<namespace-name>.svc.cluster.local, which means that if a container only uses <service-name>, it will resolve to the service which is local to a namespace. This is useful for using the same configuration across multiple namespaces such as Development, Staging and Production. If you want to reach across namespaces, you need to use the fully qualified domain name (FQDN).
Containers:
Each node in a Kubernetes cluster runs the containers that form the Pods assigned to that node. Containers in a Pod are co-located and co-scheduled to run on the same node
Container image:
Image names
Container images are usually given a name such as pause, example/mycontainer, or kube-apiserver. Images can also include a registry hostname; for example: fictional.registry.example/imagename, and possibly a port number as well; for example: fictional.registry.example:10443/imagename.
If you don’t specify a registry hostname, Kubernetes assumes that you mean the Docker public registry.
Container runtimes
The container runtime is the software that is responsible for running containers.
Pods in a Kubernetes cluster are used in two main ways:
- Pods that run a single container. The “one-container-per-Pod” model is the most common Kubernetes use case; in this case, you can think of a Pod as a wrapper around a single container; Kubernetes manages Pods rather than managing the containers directly.
- Pods that run multiple containers that need to work together. A Pod can encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers form a single cohesive unit of service—for example, one container serving data stored in a shared volume to the public, while a separate sidecar container refreshes or updates those files. The Pod wraps these containers, storage resources, and an ephemeral network identity together as a single unit.
- Note: Grouping multiple co-located and co-managed containers in a single Pod is a relatively advanced use case. You should use this pattern only in specific instances in which your containers are tightly coupled.(airflow – etc)
Each Pod is meant to run a single instance of a given application. If you want to scale your application horizontally (to provide more overall resources by running more instances), you should use multiple Pods, one for each instance. In Kubernetes, this is typically referred to as replication. Replicated Pods are usually created and managed as a group by a workload resource and its controller.
Using init containers
Because init containers have separate images from app containers, they have some advantages for start-up related code:
- Init containers can contain utilities or custom code for setup that are not present in an app image. For example, there is no need to make an image
FROManother image just to use a tool likesed,awk,python, ordigduring setup. - The application image builder and deployer roles can work independently without the need to jointly build a single app image.
- Init containers can run with a different view of the filesystem than app containers in the same Pod. Consequently, they can be given access to Secrets that app containers cannot access.
- Because init containers run to completion before any app containers start, init containers offer a mechanism to block or delay app container startup until a set of preconditions are met. Once preconditions are met, all of the app containers in a Pod can start in parallel.
- Init containers can securely run utilities or custom code that would otherwise make an app container image less secure. By keeping unnecessary tools separate you can limit the attack surface of your app container image
Example:
https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
two init-containers need to in runing status before the app container
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app.kubernetes.io/name: MyApp
spec:
containers:
- name: myapp-container
image: busybox:1.28
command: ['sh', '-c', 'echo The app is running! && sleep 3600']
initContainers:
- name: init-myservice
image: busybox:1.28
command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"]
- name: init-mydb
image: busybox:1.28
command: ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"]Working with Pods
Link: https://kubernetes.io/docs/concepts/workloads/pods/
You’ll rarely create individual Pods directly in Kubernetes—even singleton Pods. This is because Pods are designed as relatively ephemeral, disposable entities. When a Pod gets created (directly by you, or indirectly by a controller), the new Pod is scheduled to run on a Node in your cluster. The Pod remains on that node until the Pod finishes execution, the Pod object is deleted, the Pod is evicted for lack of resources, or the node fails.
Note: Restarting a container in a Pod should not be confused with restarting a Pod. A Pod is not a process, but an environment for running container(s). A Pod persists until it is deleted.
Workload
Deployments
ReplicaSet :(the purpose to have replicaset?)
https://www.kubermatic.com/blog/introduction-to-kubernetes-replicasets/
A ReplicaSet is a process that runs multiple instances of a Pod and keeps the specified number of Pods constant. Its purpose is to maintain the specified number of Pod instances running in a cluster at any given time to prevent users from losing access to their application when a Pod fails or is inaccessible.
ReplicaSet helps bring up a new instance of a Pod when the existing one fails, scale it up when the running instances are not up to the specified number, and scale down or delete Pods if another instance with the same label is created. A ReplicaSet ensures that a specified number of Pod replicas are running continuously and helps with load-balancing in case of an increase in resource usage.
StatefulSet
https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
1.Headless Services: clusterIP:None
clusterIP: None
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- port: 80
name: web
clusterIP: None
selector:
app: nginx
ClusterIP provides a single IP address for the set of Pods the Service is pointing to. This IP address is accessible only within the cluster.
Job: usecase?
The Job object can be used to support reliable parallel execution of Pods. The Job object is not designed to support closely-communicating parallel processes, as commonly found in scientific computing. It does support parallel processing of a set of independent but related work items. These might be emails to be sent, frames to be rendered, files to be transcoded, ranges of keys in a NoSQL database to scan, and so on.
Note: For those platforms that support Pods running in the host network (e.g. Linux), when pods are attached to the host network of a node they can still communicate with all pods on all nodes without NAT.
Kubernetes IP addresses exist at the Pod scope – containers within a Pod share their network namespaces – including their IP address and MAC address. This means that containers within a Pod can all reach each other’s ports on localhost. This also means that containers within a Pod must coordinate port usage, but this is no different from processes in a VM. This is called the “IP-per-pod” model.
Kubernetes networking addresses four concerns:
- Containers within a Pod use networking to communicate via loopback.
- Cluster networking provides communication between different Pods.
- The Service API lets you expose an application running in Pods to be reachable from outside your cluster.
- Ingress provides extra functionality specifically for exposing HTTP applications, websites and APIs.
- You can also use Services to publish services only for consumption inside your cluster.
Example to understand the services:
Service: A Kubernetes Service that identifies a set of Pods using label selectors.
https://kubernetes.io/docs/tutorials/services/connect-applications-service/
Expose the services:
Exposing the Service
For some parts of your applications you may want to expose a Service onto an external IP address. Kubernetes supports two ways of doing this: NodePorts and LoadBalancers. The Service created in the last section already used NodePort, so your nginx HTTPS replica is ready to serve traffic on the internet if your node has a public IP.
A Kubernetes Service is an abstraction which defines a logical set of Pods running somewhere in your cluster, that all provide the same functionality. When created, each Service is assigned a unique IP address (also called clusterIP). This address is tied to the lifespan of the Service, and will not change while the Service is alive. Pods can be configured to talk to the Service, and know that communication to the Service will be automatically load-balanced out to some pod that is a member of the Service.
This specification will create a Service which targets TCP port 80 on any Pod with the run: my-nginx label, and expose it on an abstracted Service port (targetPort: is the port the container accepts traffic on, port: is the abstracted Service port, which can be any port other pods use to access the Service). View Service API object to see the list of supported fields in service definition. Check your Service:
Sunset and Sunrise 