Friday, July 30, 2021

What is LuaRock

LuaRocks.org is a community run rock and rockspec hosting service for Lua modules. It provides an easy way to upload rocks and rockspecs compatible with the LuaRocks package manager.


Anyone can join and upload a Lua module. Modules are places in manifests, centralized lists of packages that LuaRocks can install from.

The root manifest is the global manifest on LuaRocks.org. It’s a single namespace containing all the packages that can be installed right from LuaRocks with no additional configuration. In addition to the root manifest, all accounts have their own manifests with modules uploaded by the account, and anyone can create a custom manifest for organizing collections of modules.


Throughout the lifetime of a module, there will probably be multiple versions created. Each version gets its own .rockspec file. A file containing the version number, metadata and build instructions.

Optionally, many .rock files may be built for a single rockspec. A rock is a zip file containing the rockspec and all files needed to install the module. Some rocks may be built for a specific platform (Linux, OSX, Windows, etc). The platform of a rock is identified by its filename.

For example, the following rockspec may result in the following rocks:

  • enet-1.0-0.rockspec
    • enet-1.0-0.src.rock
    • enet-1.0-0.win32-x86.rock

LuaRocks.org is a website for hosting all of these kinds of files.

Some module versions may be marked as development. These versions are placed in a development version of their regular manifest so they won’t be installed by default. This site will automatically identify a module version as development by it’s name. The owner of the module can override the classification if it was miscategorized.



How LuaRocks Works

When you tell LuaRocks to install a package you typically do something like this:

$ luarocks install enet

LuaRocks will download a manifest from each of the manifest servers it has been configured to look at. A manifest is a list of modules that a server has available. You can see the LuaRocks root manifest by going to: https://luarocks.org/manifest. It will then try to find the module you searched for that best matches your platform and the version you want (the most recent by default).

After finding the right match, it will ask the server with that module for either a .rock or .rockspec which is needed to install locally. For example, if our search was using LuaRocks.org, it might find this .rockspec if no rocks were available: https://luarocks.org/enet-1.0-0.rockspec

After downloading the right file, LuaRocks will then perform the installation and the module is ready for use.





References:

https://luarocks.org/about

Django Overriding Django Admin Templates

We can see the templates used by Django  in the admin by looking inside the Django package in your virtual environment


.../site-packages/django/contrib/admin/templates/



The admin templates come in two directories:

  1. admin is for the model object pages.
  2. registration is for password changes and logging in and out.


Just below can be done to override



{% extends "admin/change_list.html" %}

{% load i18n admin_urls %}

{% block result_list %}

<footer style="color:blue;">

    <p>The total review upto date: {{ total_review_upto_date}}<br>

    </p>

    <p>The total review to be updated: {{ total_review_tobe_updated}}<br></p>

    <p>Total review to be added : {{ total_review_tobe_added }}</p>

</footer>


{% endblock %}

Friday, July 23, 2021

Django admin customizations

By default, the Django admin displays each object by calling str() on it. You can make this screen a little more helpful by adding a .__str__() method to the Person class in core/models.py:



class Person(models.Model):

    last_name = models.TextField()

    first_name = models.TextField()

    courses = models.ManyToManyField("Course", blank=True)


    def __str__(self):

        return f"{self.last_name}, {self.first_name}"



Most of the customization you can do with the Django admin is done by modifying ModelAdmin, and you sure can modify it!


ModelAdmin has over thirty attributes and almost fifty methods. You can use each one of these to fine-tune the admin’s presentation and control your objects’ interfaces. Every one of these options is described in detail in the documentation.

To top it all off, the admin is built using Django’s templating interface. The Django template mechanism allows you to override existing templates, and because the admin is just another set of templates, this means you can completely change its HTML.



The Django admin is split into three major areas:

  1. App index
  2. Change lists
  3. Change forms


The app index lists your registered models. A change list is automatically created for each registered model and lists the objects for that model. When you add or edit one of those objects, you do so with a change form.



Modifying a Change List Using list_display


You can customize change list pages in far more ways than just modifying an object’s string representation. The list_display attribute of an admin.ModelAdmin object specifies what columns are shown in the change list. This value is a tuple of attributes of the object being modeled. For example, in core/admin.py, modify PersonAdmin as follows:


@admin.register(Person)

class PersonAdmin(admin.ModelAdmin):

    list_display = ("last_name", "first_name")


The two columns are clickable, allowing you to sort the page by the column data. The admin also respects the ordering attribute of a Meta section:


class Person(models.Model):

    # ...


    class Meta:

        ordering = ("last_name", "first_name")


    # ...


Adding the ordering attribute will default all queries on Person to be ordered by last_name then first_name. Django will respect this default order both in the admin and when fetching objects.


The list_display tuple can reference any attribute of the object being listed. It can also reference a method in the admin.ModelAdmin itself. Modify PersonAdmin again:


@admin.register(Person)

class PersonAdmin(admin.ModelAdmin):

    list_display = ("last_name", "first_name", "show_average")


    def show_average(self, obj):

        from django.db.models import Avg

        result = Grade.objects.filter(person=obj).aggregate(Avg("grade"))

        return result["grade__avg"]



By default, only those columns that are object attributes are sortable. show_average() is not. This is because sorting is performed by an underlying QuerySet, not on the displayed results.


The title for the column is based on the name of the method. You can alter the title by adding an attribute to the method:


def show_average(self, obj):

    result = Grade.objects.filter(person=obj).aggregate(Avg("grade"))

    return result["grade__avg"]


show_average.short_description = "Average Grade"



By default, Django protects you from HTML in strings in case the string is from user input. To have the display include HTML, you must use format_html():


def show_average(self, obj):

    from django.utils.html import format_html


    result = Grade.objects.filter(person=obj).aggregate(Avg("grade"))

    return format_html("<b><i>{}</i></b>", result["grade__avg"])


show_average.short_description = "Average"



Providing Links to Other Object Pages


from django.urls import reverse

from django.utils.http import urlencode


@admin.register(Course)

class CourseAdmin(admin.ModelAdmin):

    list_display = ("name", "year", "view_students_link")


    def view_students_link(self, obj):

        count = obj.person_set.count()

        url = (

            reverse("admin:core_person_changelist")

            + "?"

            + urlencode({"courses__id": f"{obj.id}"})

        )

        return format_html('<a href="{}">{} Students</a>', url, count)


    view_students_link.short_description = "Students"



Adding Filters to the List Screen


@admin.register(Course)

class CourseAdmin(admin.ModelAdmin):

    list_display = ("name", "year", "view_students_link")

    list_filter = ("year", )

# ...



Adding Search to the List Screen


Filters aren’t the only way to reduce the amount of data on the screen. Django admin also supports searching through the search_fields option, which adds a search box to the screen. You set it with a tuple containing the names of fields to be used for constructing a search query in the database.


Anything the user types in the search box is used in an OR clause of the fields filtering the QuerySet. By default, each search parameter is surrounded by % signs, meaning if you search for r, then any word with an r inside will appear in the results. You can be more precise by specifying a __ modifier on the search field.


@admin.register(Person)

class PersonAdmin(admin.ModelAdmin):

    search_fields = ("last_name__startswith", )



Changing How Models Are Edited


You can control which fields are included, as well as their order, by editing the fields option. Modify your PersonAdmin object, adding a fields attribute:

@admin.register(Person)

class PersonAdmin(admin.ModelAdmin):

    fields = ("first_name", "last_name", "courses")

# ...




ModelAdmin.get_form() is responsible for creating the ModelForm for your object. You can override this method to change the form. Add the following method to PersonAdmin:


def get_form(self, request, obj=None, **kwargs):

    form = super().get_form(request, obj, **kwargs)

    form.base_fields["first_name"].label = "First Name (Humans only!):"

    return form



If you don’t like the ModelForm that the Django admin created for you, then you can use the form attribute to register a custom form. Make the following additions and changes to core/admin.py:



from django import forms


class PersonAdminForm(forms.ModelForm):

    class Meta:

        model = Person

        fields = "__all__"


    def clean_first_name(self):

        if self.cleaned_data["first_name"] == "Spike":

            raise forms.ValidationError("No Vampires")


        return self.cleaned_data["first_name"]


@admin.register(Person)

class PersonAdmin(admin.ModelAdmin):

    form = PersonAdminForm





References:

https://realpython.com/customize-django-admin-python/


Thursday, July 22, 2021

Django Multiple admins

You can create multiple instances of the admin site on the same Django-powered website. Create multiple instances of AdminSite and place each one at a different URL.

In this example, the URLs /basic-admin/ and /advanced-admin/ feature separate versions of the admin site – using the AdminSite instances myproject.admin.basic_site and myproject.admin.advanced_site, respectively:



# urls.py

from django.urls import path

from myproject.admin import advanced_site, basic_site


urlpatterns = [

    path('basic-admin/', basic_site.urls),

    path('advanced-admin/', advanced_site.urls),

]



AdminSite instances take a single argument to their constructor, their name, which can be anything you like. This argument becomes the prefix to the URL names for the purposes of reversing them. This is only necessary if you are using more than one AdminSite.


Adding views to admin sites

Just like ModelAdminAdminSite provides a get_urls() method that can be overridden to define additional views for the site. To add a new view to your admin site, extend the base get_urls() method to include a pattern for your new view.

Note

Any view you render that uses the admin templates, or extends the base admin template, should set request.current_app before rendering the template. It should be set to either self.name if your view is on an AdminSite or self.admin_site.name if your view is on a ModelAdmin.



References:

https://docs.djangoproject.com/en/3.0/ref/contrib/admin/#multiple-admin-sites-in-the-same-urlconf

Nginx Passing a Request to a Proxied Server

When NGINX proxies a request, it sends the request to a specified proxied server, fetches the response, and sends it back to the client. It is possible to proxy requests to an HTTP server (another NGINX server or any other server) or a non-HTTP server (which can run an application developed with a specific framework, such as PHP or Python) using a specified protocol. Supported protocols include FastCGI , uwsgi , SCGI , and memcached .

To pass a request to an HTTP proxied server, the <code>proxy_pass</code> directive is specified inside a <code>location</code> . For example:



location /some/path/ {

    proxy_pass http://www.example.com/link/;

}


This example configuration results in passing all requests processed in this location to the proxied server at the specified address. This address can be specified as a domain name or an IP address. The address may also include a port:



location ~ \.php {

    proxy_pass http://127.0.0.1:8000;

}


Note that in the first example above, the address of the proxied server is followed by a URI, /link/. If the URI is specified along with the address, it replaces the part of the request URI that matches the location parameter. For example, here the request with the /some/path/page.html URI will be proxied to http://www.example.com/link/page.html. If the address is specified without a URI, or it is not possible to determine the part of URI to be replaced, the full request URI is passed (possibly, modified).

To pass a request to a non-HTTP proxied server, the appropriate **_pass directive should be used:

Note that in these cases, the rules for specifying addresses may be different. You may also need to pass additional parameters to the server (see the reference documentation for more detail).

The <code>proxy_pass</code> directive can also point to a named group of servers. In this case, requests are distributed among the servers in the group according to the specified method .


Passing Request Headers


By default, NGINX redefines two header fields in proxied requests, “Host” and “Connection”, and eliminates the header fields whose values are empty strings. “Host” is set to the $proxy_host variable, and “Connection” is set to close.

To change these setting, as well as modify other header fields, use the <code>proxy_set_header</code> directive. This directive can be specified in a <code>location</code> or higher. It can also be specified in a particular <code>server</code> context or in the <code>http</code> block. For example:



location /some/path/ {

    proxy_set_header Host $host;

    proxy_set_header X-Real-IP $remote_addr;

    proxy_pass http://localhost:8000;

}


In this configuration the “Host” field is set to the $host variable.

To prevent a header field from being passed to the proxied server, set it to an empty string as follows:



location /some/path/ {

    proxy_set_header Accept-Encoding "";

    proxy_pass http://localhost:8000;

}




References:

https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/

Django Channels with nginx, Docker

This example shows how django channels can work behind nginx


Run docker-compose up --build -d, omitting -d if you want to be fed the logs immediately (remember ctrl + z will exit out of logs without killing the containers).


Then run docker-compose run django_asgi python manage.py migrate for the initial migration. This can also be achieved by entering the container via docker exec -it django_asgi bash and then running the commands.

and finally run docker-compose run django_asgi python manage.py createsuperuser


uWSGI is a software application that "aims at developing a full stack for building hosting services". It is named after the Web Server Gateway Interface, which was the first plugin supported by the project. uwsgi is the native binary protocol that uWSGI uses to communicate with other servers


References:

https://github.com/jhnoor/docker-nginx-django-channels




How nginx processes the request 


nginx first decides which server should process the request. Let’s start with a simple configuration where all three virtual servers listen on port *:80:

server {

    listen      80;

    server_name example.org www.example.org;

    ...

}


server {

    listen      80;

    server_name example.net www.example.net;

    ...

}


server {

    listen      80;

    server_name example.com www.example.com;

    ...

}



In this configuration nginx tests only the request’s header field “Host” to determine which server the request should be routed to. If its value does not match any server name, or the request does not contain this header field at all, then nginx will route the request to the default server for this port. In the configuration above, the default server is the first one — which is nginx’s standard default behaviour. It can also be set explicitly which server should be default, with the default_server parameter in the listen directive:

server {

    listen      80 default_server;

    server_name example.net www.example.net;

    ...

}


The default_server parameter has been available since version 0.8.21. In earlier versions the default parameter should be used instead.

Note that the default server is a property of the listen port and not of the server name. More about this later.

How to prevent processing requests with undefined server names

If requests without the “Host” header field should not be allowed, a server that just drops the requests can be defined:

server {

    listen      80;

    server_name "";

    return      444;

}

Here, the server name is set to an empty string that will match requests without the “Host” header field, and a special nginx’s non-standard code 444 is returned that closes the connection.

Since version 0.8.48, this is the default setting for the server name, so the server_name "" can be omitted. In earlier versions, the machine’s hostname was used as a default server name.


Mixed name-based and IP-based virtual servers

Let’s look at a more complex configuration where some virtual servers listen on different addresses:

server {

    listen      192.168.1.1:80;

    server_name example.org www.example.org;

    ...

}


server {

    listen      192.168.1.1:80;

    server_name example.net www.example.net;

    ...

}


server {

    listen      192.168.1.2:80;

    server_name example.com www.example.com;

    ...

}

In this configuration, nginx first tests the IP address and port of the request against the listen directives of the server blocks. It then tests the “Host” header field of the request against the server_name entries of the server blocks that matched the IP address and port. If the server name is not found, the request will be processed by the default server. For example, a request for www.example.com received on the 192.168.1.1:80 port will be handled by the default server of the 192.168.1.1:80 port, i.e., by the first server, since there is no www.example.com defined for this port.

As already stated, a default server is a property of the listen port, and different default servers may be defined for different ports:

server {

    listen      192.168.1.1:80;

    server_name example.org www.example.org;

    ...

}


server {

    listen      192.168.1.1:80 default_server;

    server_name example.net www.example.net;

    ...

}


server {

    listen      192.168.1.2:80 default_server;

    server_name example.com www.example.com;

    ...

}


A simple PHP site configuration

Now let’s look at how nginx chooses a location to process a request for a typical, simple PHP site:

server {

    listen      80;

    server_name example.org www.example.org;

    root        /data/www;


    location / {

        index   index.html index.php;

    }


    location ~* \.(gif|jpg|png)$ {

        expires 30d;

    }


    location ~ \.php$ {

        fastcgi_pass  localhost:9000;

        fastcgi_param SCRIPT_FILENAME

                      $document_root$fastcgi_script_name;

        include       fastcgi_params;

    }

}


nginx first searches for the most specific prefix location given by literal strings regardless of the listed order. In the configuration above the only prefix location is “/” and since it matches any request it will be used as a last resort. Then nginx checks locations given by regular expression in the order listed in the configuration file. The first matching expression stops the search and nginx will use this location. If no regular expression matches a request, then nginx uses the most specific prefix location found earlier.

Note that locations of all types test only a URI part of request line without arguments. This is done because arguments in the query string may be given in several ways, for example:

/index.php?user=john&page=1

/index.php?page=1&user=john

Besides, anyone may request anything in the query string:

/index.php?page=1&something+else&user=john


Now let’s look at how requests would be processed in the configuration above:

  • A request “/logo.gif” is matched by the prefix location “/” first and then by the regular expression “\.(gif|jpg|png)$”, therefore, it is handled by the latter location. Using the directive “root /data/www” the request is mapped to the file /data/www/logo.gif, and the file is sent to the client.
  • A request “/index.php” is also matched by the prefix location “/” first and then by the regular expression “\.(php)$”. Therefore, it is handled by the latter location and the request is passed to a FastCGI server listening on localhost:9000. The fastcgi_param directive sets the FastCGI parameter SCRIPT_FILENAME to “/data/www/index.php”, and the FastCGI server executes the file. The variable $document_root is equal to the value of the root directive and the variable $fastcgi_script_name is equal to the request URI, i.e. “/index.php”.
  • A request “/about.html” is matched by the prefix location “/” only, therefore, it is handled in this location. Using the directive “root /data/www” the request is mapped to the file /data/www/about.html, and the file is sent to the client.
  • Handling a request “/” is more complex. It is matched by the prefix location “/” only, therefore, it is handled by this location. Then the index directive tests for the existence of index files according to its parameters and the “root /data/www” directive. If the file /data/www/index.html does not exist, and the file /data/www/index.php exists, then the directive does an internal redirect to “/index.php”, and nginx searches the locations again as if the request had been sent by a client. As we saw before, the redirected request will eventually be handled by the FastCGI server.



References:

http://nginx.org/en/docs/http/request_processing.html