You need to upgrade JDK to the required version.
You can get needed JDK version https://www.oracle.com/java/technologies/downloads/#java21
Download the deb file
wget https://download.oracle.com/java/21/latest/jdk-21_linux-x64_bin.deb
sudo dpkg -i downloaded_deb_file.deb
Pick the correct JDK version
sudo update-alternatives --config java
Do you know what is :: operator in python. You might have encountered the trick to reverse the list elements like my_list[::-1]. But ever wondered what is actually this :: operator his.
Do you know what is :: operator in python. You might have encountered the trick to reverse the list elements like my_list[::-1]. But ever wondered what is actually this :: operator his.
To a surprise it is just list slicing operator like you do my_list[1:4] to get list items from 1st position to 3rd position (4th excluding.. )
It is this only, syntax, my_list[start:end:step] and if you do not specify the start, end and the step it becomes my_list[::], which returns the entire list.
when the step is -1, it returns list in reverse order i.e.
my_list = ['c', 'a', 't']
my_list[::-1] # ['t', 'a', 'c']
Byee,…
Get current working directory
| import os | |
| os.getcwd() |
List contents of directory
| import os | |
| dir_path=os.getcwd() | |
| os.listdir(dir_path) |
List contents of directory – recursively from top to down in directory tree
| import os | |
| for root, dirs, files in os.walk(os.getcwd()): | |
| print('DIR ROOT ==>', root) | |
| print('Directories ==>', dirs) | |
| print('Files ==>', files) | |
Get name and path of current file (executing)
| import os | |
| # File Name | |
| print(__file__) | |
| # File Path | |
| print(os.path.realpath(__file__)) |
Get filename from path and get parent folder from path
| import os | |
| file_path='/home/sachin/test.txt' | |
| # Get File Name | |
| print(os.path.basename(file_path)) # test.txt | |
| # Get Dir of file | |
| print(os.path.dirname(file_path)) # sachin | |
| # Another way to get Dir of the file | |
| for root, dirs, files in os.walk(file_path): | |
| print('DIR ROOT ==>', root) | |
| break |
Get root folder of the path
| import os | |
| path='/home/sachin/test.txt' | |
| print(os.path.realpath(path).split(os.path.sep)[0]) # home | |
Copy file in the same folder with another name
| import os | |
| import datetime | |
| original_file_path = __file__ | |
| copied_file_path = 'copy_' + datetime.datetime.now().strftime('%d_%m_%Y_%H_%M_%S') + '_'+os.path.basename(original_file_path) | |
| with open(original_file_path, 'rb') as original_file: | |
| with open(os.path.join(os.path.dirname(original_file_path), copied_file_path), 'wb') as copied_file: | |
| copied_file.write(original_file.read()) | |
Rename file, Move file from one folder to another folder, Delete file, Delete folder
| import os | |
| file_path = '/home/foo/baz.txt' | |
| # Rename | |
| os.rename(file_path, 'test_%s' % os.path.basename(file_path)) | |
| # Move file from one folder to another folder | |
| dest_path = '/home/foo/bar.txt' | |
| os.rename(file_path, dest_path) | |
| # Delete file | |
| file_path = '/home/foo/file.txt' | |
| os.remove(file_path) | |
| # Delete folder | |
| dir_path = '/home/foo/bar/' | |
| os.rmdir(dir_path) | |
Check exists file, directory , either
| import os | |
| # Check if file exists | |
| os.path.isfile('test.txt') | |
| # Check if folder exists | |
| os.path.isdir('test.txt') | |
| # Check if file or dir exists | |
| os.path.exists('test.txt') | |
Check permissions
| import os | |
| # Check if file or directory writable | |
| os.access("myfile", os.W_OK) | |
| # Check if file or directory readable | |
| os.access("myfile", os.R_OK) | |
| # Check if file or directory executable | |
| os.access("myfile", os.X_OK) |
Get file or directory modified time, created time
| import os | |
| # Get last modified time of file or dir | |
| os.path.getmtime('myfile.txt') | |
| # Get created time | |
| os.path.getctime('myfile.txt') |
REST APIs can be created elegantly using python’s djangorestframework module in a Django project.
Django Rest Framework module provides simple and sophisticated ways of creating APIs. Django Rest Framework is
also known as DRF, we will use this acronym to save space and time.
Following prerequisites required
1. Install “djangorestframework” module in your django project.
`pip install djangorestframework`
2. Add “rest_framework” to INSTALLED_APPS list in settings.py
3. Run the migrations
`python manage.py makemigrations`
`python manage.py migrate`
Now setup is completed, lets create simple API which returns sample data in JSON format.
We will go from simple to complex, We will create simple API which gives you JSON output, just a message “Hello
World”. Once completed, you check output in your browser at http://localhost:8000/hello-world-api/
Fastest (but not dirty) way
The shortest and fastest way to is to create a python function and write all logic there.
In views.py file add following
| from rest_framework.decorators import api_view | |
| from rest_framework.response import Response | |
| @api_view() | |
| def hello_world_api(request): | |
| return Response({"message": "Hello, world!"}) |
Add link to urls.py file
| from django.contrib import admin | |
| from django.urls import path | |
| from drf_api.views import hello_world_api | |
| urlpatterns = [ | |
| path('hello-world-api/', hello_world_api), # Add this to add API to project urls | |
| path('admin/', admin.site.urls), | |
| ] |
Somewhat more elegant way.
We can create python class which extends APIView.
| from rest_framework.views import APIView | |
| from rest_framework.response import Response | |
| class HelloWorldAPIView(APIView): | |
| def get(self, request): | |
| return Response({'message': 'Hello, World!'}) |
To use above as API you will need to connect it to URL to do that,
in project’s `urls.py` file add following entry in `urlpatterns` list.
| from django.contrib import admin | |
| from django.urls import path | |
| from drf_api.views import HelloWorldAPIView | |
| urlpatterns = [ | |
| path('hello-world-api/', HelloWorldAPIView.as_view()), | |
| path('admin/', admin.site.urls), | |
| ] |
Specialized Views, don’t get stuck into “Hello World” only
There are classes in DRF which you can extend to create APIs for specialized cases, like listing database
table entries, creating/modifying/deleting database table record. You can do things close to zen of
REST APIs.
You can create REST API around Django model which is database table under the hood. Almost in all cases model corresponds to table in database. Here you can think of this model as “resource” in REST terminology, and on this resource you can create apis to honor REST verbs like GET, POST, PUT, DELETE etc.
DRF provides lot of model based views which gives you built in APIs to manage model. To use model based APIs you will need model
| from django.db import models | |
| class Movie(models.Model): | |
| title = models.CharField(max_length=100) | |
| director = models.CharField(max_length=100) |
and model serializer
| from rest_framework import serializers | |
| from .models import Movie | |
| class MovieSerializer(serializers.ModelSerializer): | |
| class Meta: | |
| model = Movie | |
| fields = '__all__' |
Lets start with ListAPIView, Classes you can extend are
ListAPIView
This gives you GET API to list model items.
| from rest_framework.generics import ListAPIView | |
| from .models import Movie | |
| from .serializers import MovieSerializer | |
| class MovieListAPIView(ListAPIView): | |
| queryset = Movie.objects.all() | |
| serializer_class = MovieSerializer |
You will need to add URL entry in project URLs,
| from django.contrib import admin | |
| from django.urls import path | |
| from drf_api.movie_list_api_view import MovieListAPIView | |
| urlpatterns = [ | |
| path('movie-list/', MovieListAPIView.as_view()), | |
| path('admin/', admin.site.urls), | |
| ] |
You can check http://localhost:8000/movie-list/, Response for API will be,
| [ | |
| { | |
| "id": 1, | |
| "title": "The Shawshank Redemption", | |
| "director": "Frank Darabont" | |
| }, | |
| { | |
| "id": 2, | |
| "title": "The Godfather", | |
| "director": "Francis Ford Coppola" | |
| }, | |
| { | |
| "id": 3, | |
| "title": "The Dark Knight", | |
| "director": "Christopher Nolan" | |
| } | |
| ] |
We will not list code samples for rest of the views here, but I have added them on github project
CreateAPIView
This POST api to create model object. e.g. curl -d “{\”title\”: \”New Movie 2\”, \”director\”:\”New Director 2\”}” -H “Content-Type: application/json”
-X POST http://localhost:8000/movie-create-api/ This will create new model objects. It supports POST method only.
RetrieveAPIView
This gives you GET API(accepts pk of model as param) to give particular model item. e.g. http://localhost:8000/movie-retrieve-api/1/ , This will fetch and display movie record of id equal to 1. If you give invalid id, API will return empty response with 404 status.
DestroyAPIView
This gives you DELETE API(accepts pk of model as param) to delete particular model item, e.g. curl -X DELETE http://localhost:8000/movie-destroy-api/3/, This will delete movie with id 3. It accepts DELETE as http request method.
UpdateAPIView
This gives you PUT API (accepts pk of model as param) to update particular model item. It accepts PUT and PATCH as http request methods. For PUT http method it needs you send whole object to update.
e.g. curl -d “{\”title\”: \”The Shawshank Redemption 2\”, \”director\”:\”Frank Darabont I\”}” -H “Content-Type:
application/json” -X PUT http://localhost:8000/movie-update-api/1/
For PATCH http method you can send individual attribute.
e.g. curl -d “{\”title\”: \”The Shawshank Redemption 3\”}” -H “Content-Type: application/json” -X PATCH http://l
ocalhost:8000/movie-update-api/1/
RetrieveUpdateAPIView
This is mix of retrieve and update. GET request along with id will retrieve the model record curl -H “Content-Type: application/json” -X GET http://localhost:8000/movie-retrieve-destroy-api/1/ and PUT and PATCH requests will update the same model record.
ListCreateAPIView
This is mix of list and create. GET request to API will return list of model items curl -H “Content-Type: application/json” -X GET http://localhost:8000/movie-list-create-api/, and POST request to API will try to create new model item curl -d “{\”title\”: \”New Movie\”, \”director\”:\”New Director\”}” -H “Content-Type: application/json” -X
POST http://localhost:8000/movie-list-create-api/
.
RetrieveDestroyAPIView
This is mix of retrieve and delete. GET request to API along with id will retrieve model record curl -H “Content-Type: application/json” -X GET http://localhost:8000/movie-retrieve-destroy-api/4/
and DELETE request will delete the corresponding model record curl -H “Content-Type: application/json” -X DELETE http://localhost:8000/movie-retrieve-destroy-api/4/.
RetrieveUpdateDestroyAPIView
This is mix of retrieve, update and delete. GET request curl -H “Content-Type: application/json” -X GET http://localhost:8000/movie-retrieve-update-destroy-api/5/ will return model with given id. PUT,PATCH will update model object as whole and partially respectively. DELETE request will delete the model object of given id.
Viewsets
You can group related views in single class aka viewsets in DRF.
ViewSet
This class by default does not provide any actions, but you can define list, create, retrieve, update, partial_update and destroy methods to use it in REST convention on URL with methods GET, POST, GET, PUT, PATCH and DELETE respectively.
| from rest_framework.viewsets import ViewSet, ModelViewSet | |
| from .models import Movie | |
| from .serializers import MovieSerializer | |
| from rest_framework.response import Response | |
| from django.shortcuts import get_object_or_404 | |
| class MovieViewSet(ViewSet): | |
| def list(self, request): | |
| queryset = Movie.objects.all() | |
| serializer = MovieSerializer(queryset, many=True) | |
| return Response(serializer.data) | |
| def create(self, request): | |
| serializer = MovieSerializer(data=request.data) | |
| serializer.is_valid(raise_exception=True) | |
| serializer.save() | |
| return Response(serializer.data) | |
| def retrieve(self, request, pk=None): | |
| queryset = Movie.objects.all() | |
| movie = get_object_or_404(queryset, pk=pk) | |
| serializer = MovieSerializer(movie) | |
| return Response(serializer.data) | |
| def update(self, request, pk=None): | |
| queryset = Movie.objects.all() | |
| movie = get_object_or_404(queryset, pk=pk) | |
| serializer = MovieSerializer(movie, data=request.data) | |
| serializer.is_valid(raise_exception=True) | |
| serializer.save() | |
| return Response(serializer.data) | |
| def partial_update(self, request, pk=None): | |
| queryset = Movie.objects.all() | |
| movie = get_object_or_404(queryset, pk=pk) | |
| serializer = MovieSerializer(movie, data=request.data, partial=True) | |
| serializer.is_valid(raise_exception=True) | |
| serializer.save() | |
| return Response(serializer.data) | |
| def destroy(self, request, pk=None): | |
| queryset = Movie.objects.all() | |
| movie = get_object_or_404(queryset, pk=pk) | |
| movie.delete() | |
| return Response({}) |
In project urls file you can add link as following,
| from rest_framework.routers import DefaultRouter | |
| from drf_api.movie_viewset_api import MovieViewSet | |
| router = DefaultRouter() | |
| router.register(r'movies-viewset', MovieViewSet, basename='movies-viewset') | |
| urlpatterns = router.urls |
GenericViewSet
This gives you class where get_object and get_queryset methods are given, it does not provide any action by default If you want to use it you will need to extend one of the mixins, like CreateModelMixin, ListModelMixin, RetrieveModelMixin, UpdateModelMixin and DestroyModelMixin these will give you API to create, list, retrieve, update and delete model entries respectively. Using GenericViewSet and mixins you can have actions which suits your need.
ModelViewSet
This is mix of list, create, retrieve, update, partial update, delete views for model.
ReadOnlyModelViewSet
This is mix of list, retrieve actions for model. It does not involve any action which will update model entry, hence good for read only API endpoint for your model.
As there are lot of options available, this may be confusing sometime which one to pick. May be this can help to choose,
Do you have a single model class which can be referred as “resource” in REST way
Yes – Use specialized model based views i.e. ModelViewSet, ReadOnlyModelViewSet.
No – Use APIView and customized it to your need.
If you have multiple model class manipulations or using raw queries, but still want to create API in REST way, Use ViewSet and implement the REST methods required.
If you have use case where model is not involved or it is not core of the functionality e.g. You are uploading file processing it and returning result on the fly, you can use APIView.
What next?
Apart from view functions, classes there are serializers, permission classes you will need to know to make
maximum use of DRF toolset.
I hope this will help you to create APIs(REST) for your application. It is very simple and reliable
way, once you grok it, you will need to look back to create APIs in Django. This blog lists general
cases, there can be other specialized or complex cases where you might need to do more, but basics will
remain same. You can start your API with this basic knowledge and then extend it further to match the
requirement. If you want to know more about how particular API case can be addressed you can add comment here, I will try to get back to you.
Till then, happy coding 🙂
Here I am listing docker handy commands and information which will be helpful in using Docker. I will keep on updating this file, hence making this file as Github gist.
It lets you package your application in isolated environment (including os). Docker runs your application in light weight component called container.
docker container run hello-world
docker container ls
docker container ls --all
docker container stop <container-id>
docker container stop $(docker container ls -q)
docker container run --detach --publish 80:80 nginx:alpine
docker-machine ip
docker container run --interactive --tty ubuntu:18.04 or
docker container run -it ubuntu:18.04
Docker file is a text document which contains steps to build a docker image which will be used to create dokcer container. Default name of the file is Dockerfile. This file can contain following
FROM ubuntu
RUN apt-get update && apt-get install nano
FROM Specifies the base image that your image will start from. It is only the required instruction to build image.
RUN Specifies the commands which will get executed one by one as part of image build process. e.g. RUN apt-get install -y git
ENV Sets environment variable e.g. ENV gituser=”sachin”
COPY Copy files from build context into image. e.g. COPY . /app
EXPOSE expose the ports from container e.g EXPOSE 8000
VOLUME Creates directory inside image which can be mapped to external storage. Value can be JSON array. e.g. VOLUME [“/var/log”]
CMD Specifis command to run when container starts e.g CMD [“python”,”manage.py”,”runserver”, ‘0:8000″]. Format is CMD [“executable”,”param1″,”param2″]. All instructions gets executed in order from docker file to build image except one CMD instruction, which gets executed when container is created.
docker image build –tag : .
e.g.
docker image build –tag docker-notes:v1.0 .
Here . is the build context, docker client will send(copy) contents of the build context to server and server stores it in working directory and uses it to build the image.
If you dont specify tag, docker takes ‘latest’ as default tag.
docker image ls
Notes
Docker adds entry in the hosts file (/etc/hosts) with ip address of the container and id of the container. So the references by container id on the same machine goes to container only.
If you have already created virtual environment and want to set the created virtual environment in PyCharm for project, you can use following. You can use Create virtual environment for python projects in Ubuntu to create virtual environment.
1. Open project in PyCharm editor.
2. Open settings , “File” > “Settings”
3. Select “Project” > “Project Interpreter” section.
4. Near “Project Interpreter” input box, click on cog icon which opens sub-menu click “Add” option.
5. In “Add Python Interpreter” window click on “Existing Environment” and browse the “Interpreter” input to python executable file (python.exe) from created environment folder.
6. Click on “Ok” , this closes “Add Python Interpreter” window.
7. Click on “Ok”, this closes “Settings” window.
8. Close the PyCharm editor and reopen it, on terminal(at the bottom of the editor) you will see the python environment activated.
You can also set folder in your project directory as root folder so that PyCharm recognizes inner modules for importing and autocomplete.
Right Click the Folder > hover “Mark Directory as” > click on “Mark as sources Root”
Installing from source
Prerequisites
sudo apt-get update
sudo apt-get install build-essential tcl
cd /tmp
curl -O http://download.redis.io/redis-stable.tar.gz
tar xzvf redis-stable.tar.gz
cd redis-stable
make
make test
sudo make install
Config for installing from source
sudo mkdir /etc/redis
sudo cp /tmp/redis-stable/redis.conf /etc/redis
sudo nano /etc/redis/redis.conf
Update above file for following line
supervised systemd
dir /var/lib/redis
For running Redis as service
sudo nano /etc/systemd/system/redis.service
Add following
[Unit]
Description=Redis In-Memory Data Store
After=network.target
[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always
[Install]
WantedBy=multi-user.target
Create Redis user and dirs
sudo adduser --system --group --no-create-home redis
sudo mkdir /var/lib/redis
sudo chown redis:redis /var/lib/redis
sudo chmod 770 /var/lib/redis
Start and Test Redis
sudo systemctl start redis
sudo systemctl status redis
Install pip
sudo apt-get install python3-pip
Install virtualenv using pip
sudo pip3 install virtualenv
Create Virtual Environment
virtualenv <virtual-env-name> -p <python-path>
Activate virtual environment
source <virtual-env-name>/bin/activate
Deactivate virtual environment
deactivate
It is just a command line interface for google cloud platform services. Another convenient way to use GCP(google clould platform) services. You can achieve same using Web Console .
You can follow Google doc to install the Google Cloud SDK. for Ubuntu its
export CLOUD_SDK_REPO=”cloud-sdk-$(lsb_release -c -s)”
echo “deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main” | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
sudo apt-get update && sudo apt-get install google-cloud-sdk
gcloud init
Google cloud commands are structured like following,
gcloud [availability-release] <command-group> [command-options]
Here value for availability-release is either empty meaning general release or values like “beta”, “alpha”, “preview” for beta, alpha and preview releases respectively.
Whereas command-group can be “compute”, “app”, “auth” etc. more on command groups here
You can get help for a command using,
gcloud <command-group> –help
Good reference is here from google about gcloud command structure and available options.
You can use mysqldiff command to compare two databases,
mysqldiff –server1=user:password@host:port –server2=user:password@host:port –difftype=sql db1:db2
Reference:
https://docs.oracle.com/cd/E17952_01/mysql-utilities-1.6-en/mysqldiff.html
TheNumberZero 