After giving an overview of the in-kernel basics of the Linuxulator, I want now to describe how to add support for new ioctls to the Linuxulator.

Where are the files to modify?

The platform independent code for the ioctls is in SRC/sys/compat/linux/linux_ioctl.c. The defines to have names for the ioctl values are in SRC/sys/compat/linux/linux_ioctl.h.

How to modify them?

First of all create a new header which will contain all the structures, named values and macros for those new ioctls. As written above, the ioctl values (e.g. #define LINUX_VIDIOC_ENCODER_CMD 0x564d /* 0xc028564d */) do not belong there, they shall be added to linux_ioctl.h. During the course of adding support for ioctls, you will need this new header. Add it in the SRC/sys/compat/linux/ directory, and prefix the name with a linux_. It would be good to decide on a common tag here (referenced as yourtag in the following), and stay with it. Use it wherever you need to have some specific name for the ioctl-set you want to add. In this case it would result in linux_yourtag.h (or even linux_ioctl_yourtag.h, depending if this is used for something very specific to the ioctls, or some generic linux feature) as the name of the header file. This was not done in the past, so do not expect that the names inside the linux_ioctl.c file will be consistent to this naming scheme, but it is never too late to correct mistakes of the past (at least in Open Source software development).

Now add this header to linux_ioctl.c (you need to include compat/linux/linux_yourtag.h). After that add the ioctl values to linux_ioctl.h. As can be seen above, the defines should be named the same as on linux, but with a LINUX_ prefix (make sure they where not defined before somewhere else). The ioctl values need to be the same hex values as in Linux, off course. Sort them according to their hex value. When you added all, you need to add two more defines. The LINUX_IOCTL_yourtag_MIN and LINUX_IOCTL_yourtag_MAX ones. The MIN-one needs to be an alias for the first (sorted according to the hex value) ioctl you added, and MAX needs to be an alias for the last (again, sorted according to the hex value) ioctl you added.

The next step is to let the Linuxulator know that it is able to handle the ioctls in the LINUX_IOCTL_yourtag_MIN to LINUX_IOCTL_yourtag_MAX range. Search the static linux_ioctl_function_t section of linux_ioctl.c and add such a variable for your ioctl set. The name of the variable should be something like linux_ioctl_yourtag.

Similar for the handler-definition for this. Search the static struct linux_ioctl_handler section and add a yourtag_handler. Set it to { linux_ioctl_yourtag, LINUX_IOCTL_yourtag_MIN, LINUX_IOCTL_yourtag_MAX }. To make this handler known to the Linuxulator, you need to add it to the DATA_SET section. Add DATA_SET(linux_ioctl_handler_set, yourtag_handler) there.

Now the meat, the function which handles the ioctls. You already defined it as linux_ioctl_function_t, but now you need to write it. The outline of it looks like this:

static int
linux_ioctl_yourtag(struct thread *td, struct linux_ioctl_args *args)
{
        struct file *fp;
        int error;
        switch (args->cmd & 0xffff) {
        case LINUX_an_easy_ioctl:
                break;
        case LINUX_a_not_so_easy_ioctl:
                /* your handling of the ioctl */
                fdrop(fp, td);
                return (error);
        /* some more handling of your ioctls */
        default:
       return (ENOIOCTL);
        }
        error = ioctl(td, (struct ioctl_args *)args);
        return (error);
}

An easy ioctl in the switch above is an ioctl where you do not have to do something but can pass the ioctl through to FreeBSD itself. The not so easy ioctl case is an ioctl where you need to do e.g. a fget(td, args->fd, &fp). This is just an example, there are also other possibilities where you need to do additional stuff before the return, or where you do not pass the ioctl to FreeBSD. A typical example of what needs to be done here is to copy values from linux structures to FreeBSD structures (and the other way too), or to translate between 64bit and 32bit. Linux programs on amd64 are 32bit executables and 32bit structures/pointers. To make this work on amd64, you need to find a way to map between the two. There are examples in the kernel where this is already the case. The more prominent examples in the 64bit<->32bit regard are the v4l and v4l2 ioctls.

The tedious part is to research if a translation has to be done and if yes what needs to be translated how. When this is done, most of the work is not so hard. The linux_yourtag.h should contain the structures you need for this translation work.

It is also possible to add ioctls in a kernel module, but this is not subject to this description (I will update this posting with a link to it when I get time to write about it).