To create new wiki account, please join us on #znc at Libera.Chat and ask admins to create a wiki account for you. You can say thanks to spambots for this inconvenience.

Modpython: Difference between revisions

From ZNC
Jump to navigation Jump to search
Update info about SWIG
 
(22 intermediate revisions by 4 users not shown)
Line 5: Line 5:


== Compiling ==
== Compiling ==
First, you need to use ./configure with option --enable-python.
First, you need to use ./configure with option --enable-python, or cmake with -DWANT_PYTHON=ON.


If you're building from [[git]], you need to have [http://www.swig.org/ SWIG] installed. If you're building from tarball (nightly or release), SWIG is not required.
If you're building from [[git]], you need to have [http://www.swig.org/ SWIG] installed (Note: Must be => 3.0.0). If you're building from tarball (nightly or release), SWIG is not required.
 
If for some reason you chose to compile python3 yourself, do it with --enable-shared option. If you use python from your distro (e.g. via apt-get, yum, etc), it is already compiled in the right way, no need to recompile anything.
 
If python was compiled without this option, you may see errors like this: <code>/usr/local/lib/znc/modpython.so: undefined symbol: forkpty</code>


== Usage ==
== Usage ==
Line 17: Line 21:


{{Module arguments|type=global}}
{{Module arguments|type=global}}
== Caveats ==
* Python multithreading doesn't work properly. However you may try multiprocessing.
* ZNC executes most of the operations (including Python modules) in single thread. It means you need to be careful not to block current thread, instead all your IRC connections may timeout. Webadmin will also stop working. Ideally you offload heavy operations to separate thread. Or actually to separate process, since multithreading doesn't work.


== Writing new python3 modules ==
== Writing new python3 modules ==
=== Basics ===
=== Basics ===


Every python module is file named like modulename.py (.pyc and .so are supported too, but I doubt that you want to write ZNC module as C python extension ;) ) and is located in usual modules directories.
Every python module is file named like modulename.py (since ZNC 1.9 you can also use the package and name it modulename/__init__.py) and is located in usual modules directories (see [[Modules#Managing_Modules|here]] for details).
The file must contain class with exactly the same name as the module itself.
The file '''must''' contain class with exactly the same name as the module itself.
The class should be derived from <code>znc.Module</code>.
The class should be derived from <code>znc.Module</code>.


Line 41: Line 49:
<code>OnShutdown</code> is called when the module is going to be unloaded.
<code>OnShutdown</code> is called when the module is going to be unloaded.


If a callback returns None, a reasonable default is substituted. If a callback raises an exception, the default value is assumed too. Note: You should return something explicitly anyway ([http://www.python.org/dev/peps/pep-0020/ PEP 20]), also such default behavior WILL be changed in some future version.
If a callback returns None or doesn't return anything instead of returning something (except for the <code>void</code> return type, of course), the behavior can be bizzare. Do not do this, even though sometimes it appears to work fine. In some future version this may be changed to check return types more strictly. The same goes for exceptions raised from the callback.


When a module callback should return CModule::EModRet, you can use values such as <code>znc.CModule.CONTINUE</code> or just <code>znc.CONTINUE</code>.
When a module callback should return CModule::EModRet, you can use values such as <code>znc.CModule.CONTINUE</code> or just <code>znc.CONTINUE</code>.


ZNC C++ API can be found [http://people.znc.in/~psychon/znc/doc/ here].
ZNC C++ API can be found [http://docs.znc.in/ here].
Most of it should just work for python modules.
Most of it should just work for python modules.
The following text describes mostly features, differences and caveats.
The following text describes mostly features, differences and caveats.
Line 66: Line 74:
     module_types = [znc.CModInfo.UserModule, znc.CModInfo.NetworkModule]
     module_types = [znc.CModInfo.UserModule, znc.CModInfo.NetworkModule]
The first element of the list is the default module type. It's used when no type is specified. So <code>/znc loadmod pyusernetworkmod</code> will load it as user module.
The first element of the list is the default module type. It's used when no type is specified. So <code>/znc loadmod pyusernetworkmod</code> will load it as user module.
=== Module metadata ===
Several settings can be configured, using the attributes like shown below:
module_types = ... (see above section)
description = "This module does this and that"
wiki_page = "my_module"
has_args = True (the default is False)
args_help_text = "The arguments are foo and bar"


=== Strings ===
=== Strings ===
Line 161: Line 179:
     success.b = False # similar to string with its .s
     success.b = False # similar to string with its .s
     return znc.HALT
     return znc.HALT
=== Message types ===
Since ZNC 1.7.0.
To convert between various subclasses of <code>CMessage</code>, use this:
<pre>
num_msg = msg.As(znc.CNumericMessage)
</pre>


=== Module's NV ===
=== Module's NV ===
Line 175: Line 201:
         for k, v = self.nv.items():
         for k, v = self.nv.items():
             ...
             ...
=== Objects ===
SWIG distinguish between instances from ZNC and instances created in Python. All instances that are created during python time are garbage collected as soon they leave scope (e.g. at the end of the function or module). To move an instance to the ZNC scope, so it can be used after the lifetime of the function/module, set the special object property .thisown to 0.
For example this is required if you add new user:
  new_user = znc.CUser(username)
  str_err = znc.String()
  if znc.CZNC.Get().AddUser(new_user, str_err):
    new_user.thisown = 0  # new_user won't be garbage collected at the end of the function anymore
Similiar if you want to free an instance (like removing a listener), you should make sure that the memory gets garbage collected
  listener = ...
  if znc.CZNC.Get().DelListener(listener):
    listener.thisown = 1
=== IRCv3 server-dependent capabilities ===
Available since ZNC 1.9. (Server-independent caps can be implemented before that, but with the same API as in C++ modules)
While in C++ you'd need to inherit from <code>CCapability</code>, here <code>self.AddServerDependentCapability()</code> accepts two callable objects:
def OnLoad(self, args, ret):
  def server_change(ircnetwork, state):
    self.PutModule('Server changed support: ' + ('true' if state else 'false'))
  def client_change(client, state):
    self.PutModule('Client changed support: ' + ('true' if state else 'false'))
  self.AddServerDependentCapability('testcap', server_change, client_change)
  return True


=== Web ===
=== Web ===
Line 224: Line 281:
* cycles - Number of times to run the <code>RunJob</code> function. 0 means infinite. Default is 1.
* cycles - Number of times to run the <code>RunJob</code> function. 0 means infinite. Default is 1.
* description - Text description of the timer. Default doesn't matter.
* description - Text description of the timer. Default doesn't matter.
* label - String identifying the timer. If the module already has an active timer with the same label, the new one will not be created. Default is <code>pytimer</code>.
  # timertest.py
  # timertest.py
  import znc
  import znc
Line 233: Line 291:
  class timertest(znc.Module):
  class timertest(znc.Module):
     def OnModCommand(self, cmd):
     def OnModCommand(self, cmd):
         timer = self.CreateTimer(testtimer, interval=4, cycles=1, description='Says "foo bar" after 4 seconds')
         timer = self.CreateTimer(testtimer, interval=4, cycles=1, description='Says "foo bar" after 4 seconds', label='moo')
         timer.msg = 'bar'
         timer.msg = 'bar'


Line 315: Line 373:
         if port > 0:
         if port > 0:
             message.s = "Listening on all IPv6 interfaces on port {0} using SSL".format(port)
             message.s = "Listening on all IPv6 interfaces on port {0} using SSL".format(port)
        return True


Use <code>Write</code> to write strings, and <code>WriteBytes</code> to write binary data.
Use <code>Write</code> to write strings, and <code>WriteBytes</code> to write binary data.
Line 335: Line 394:
If you just want to show ZNC version to humans, usually just <code>znc.CZNC.GetTag()</code> is good.
If you just want to show ZNC version to humans, usually just <code>znc.CZNC.GetTag()</code> is good.
  znc.CZNC.GetTag() # Returns, for example, 'ZNC 0.097 - http:/<nowiki></nowiki>/znc.sourceforge.net'
  znc.CZNC.GetTag() # Returns, for example, 'ZNC 0.097 - http:/<nowiki></nowiki>/znc.sourceforge.net'
Check [http://people.znc.in/~psychon/znc/doc/ ZNC C++ documentation] for details.
Check [http://docs.znc.in/ ZNC C++ documentation] for details.


For getting ZNC version, you can use read following variables:
For getting ZNC version, you can use read following variables:

Latest revision as of 11:41, 11 February 2024


Modpython allows you to use modules written on python 3.

Compiling

First, you need to use ./configure with option --enable-python, or cmake with -DWANT_PYTHON=ON.

If you're building from git, you need to have SWIG installed (Note: Must be => 3.0.0). If you're building from tarball (nightly or release), SWIG is not required.

If for some reason you chose to compile python3 yourself, do it with --enable-shared option. If you use python from your distro (e.g. via apt-get, yum, etc), it is already compiled in the right way, no need to recompile anything.

If python was compiled without this option, you may see errors like this: /usr/local/lib/znc/modpython.so: undefined symbol: forkpty

Usage

Loading and unloading of python3 modules is similar to C++ modules. For example, you can use /znc loadmod or webadmin.

If you unload modpython, all python modules are automatically unloaded too.

Arguments

This global module takes no arguments.

Read loading modules to learn more about loading modules.

Caveats

  • Python multithreading doesn't work properly. However you may try multiprocessing.
  • ZNC executes most of the operations (including Python modules) in single thread. It means you need to be careful not to block current thread, instead all your IRC connections may timeout. Webadmin will also stop working. Ideally you offload heavy operations to separate thread. Or actually to separate process, since multithreading doesn't work.

Writing new python3 modules

Basics

Every python module is file named like modulename.py (since ZNC 1.9 you can also use the package and name it modulename/__init__.py) and is located in usual modules directories (see here for details). The file must contain class with exactly the same name as the module itself. The class should be derived from znc.Module.

# pythonexample.py

import znc

class pythonexample(znc.Module):
    description = "Example python3 module for ZNC"

    def OnChanMsg(self, nick, channel, message):
        self.PutModule("Hey, {0} said {1} on {2}".format(nick.GetNick(), message.s, channel.GetName()))
        return znc.CONTINUE

All callbacks have the same name as in C++, and have the same arguments, but with reference to self before first argument, as usually in python.

def OnShutdown is used as destructor (instead of python's __del__). OnShutdown is called when the module is going to be unloaded.

If a callback returns None or doesn't return anything instead of returning something (except for the void return type, of course), the behavior can be bizzare. Do not do this, even though sometimes it appears to work fine. In some future version this may be changed to check return types more strictly. The same goes for exceptions raised from the callback.

When a module callback should return CModule::EModRet, you can use values such as znc.CModule.CONTINUE or just znc.CONTINUE.

ZNC C++ API can be found here. Most of it should just work for python modules. The following text describes mostly features, differences and caveats.

Module types

Before ZNC 0.207 only user python modules are supported. Since ZNC 0.207 network, user and global modules are supported. By default every module is network-only.

If you want to make your module accessible only at user level, use this:

class pyusermod(znc.Module):
    module_types = [znc.CModInfo.UserModule]

For global modules use this:

class pyglobalmod(znc.Module):
    module_types = [znc.CModInfo.GlobalModule]

If you want to make your module to be loadable as both user module and network module, use this:

class pyusernetworkmod(znc.Module):
    module_types = [znc.CModInfo.UserModule, znc.CModInfo.NetworkModule]

The first element of the list is the default module type. It's used when no type is specified. So /znc loadmod pyusernetworkmod will load it as user module.

Module metadata

Several settings can be configured, using the attributes like shown below:

module_types = ... (see above section)
description = "This module does this and that"
wiki_page = "my_module"
has_args = True (the default is False)
args_help_text = "The arguments are foo and bar"

Strings

All ZNC classes are accessible from python with znc. prefix. The exception is CString. All uses of CString by value is transparently translated to/from python string objects.

// C++
void Foo(const CString& s);
# python
znc.Foo("bar")

The same for case where you get CString by value:

// C++
class CModule {
    ...
    virtual bool OnLoad(const CString& sArgsi, CString& sMessage);
};
# python
class foo(znc.Module):
    def OnLoad(self, args, message):
        if args == "bar":
            return True
        return False

If you need to use CString by reference, use class znc.String and its attribute s:

// C++
void Foo(CString& s) {
    s = "bar";
}
# python
x = znc.String()
znc.Foo(x);
print(x.s); # prints 'bar' to stdout

The same if you get CString& as argument:

// C++
class CModule {
    ...
    virtual bool OnLoad(const CString& sArgsi, CString& sMessage);
};
# python
class foo(znc.Module):
    def OnLoad(self, args, message):
        message.s = 'bar'
        return True

Note: don't try to use the string which you got in the overloaded method for calls to other methods

# C++
class CModule {
    virtual void OnFoo(CString& sMsg);
    void Bar(CString& sMsg); // appends "Bar" to sMsg
}
# python, wrong way
class foo(znc.Module):
    def OnFoo(self, msg):
        self.Bar(msg) # ZNC crashes here
# python
class foo(znc.Module):
    def OnFoo(self, msg):
        s = znc.String()
        s.s = msg.s # so that old value is preserved
        self.Bar(s)
        msg.s = s.s # put result back to msg

So, you want to override a hook void OnFoo(const CString& sBar) and you actually don't want to write to sBar. So you probably will want to use the argument as python string. But, let's assume that in next ZNC version OnFoo's signature will be changed to void OnFoo(CString& sBar). This will break your module! To be on safe side, convert the argument to string with usual str() function. (Note: support for str() was added in ZNC 0.099)

// C++
class CModule {
    ...
    virtual bool OnLoad(const CString& sArgs, CString& sMessage);
    virtual EModRet OnRaw(CString& sLine);
};
# python
class foo(znc.Module):
    def OnLoad(self, args, message):
        if str(args).startswith('bar'):
            ...
    def OnRaw(self, line):
        if str(line).startswith('bar'):
            ...

Booleans

Moost booleans just work. If a callback gets bool& as a parameter, use this:

// C++
virtual EModRet OnModuleLoading(const CString& sModName, const CString& sArgs,
    CModInfo::EModuleType eType, bool& bSuccess, CString& sRetMsg);
# python
def OnModuleLoading(self, name, args, typ, success, retmsg):
    success.b = False # similar to string with its .s
    return znc.HALT

Message types

Since ZNC 1.7.0.

To convert between various subclasses of CMessage, use this:

num_msg = msg.As(znc.CNumericMessage)

Module's NV

module.nv is a dict-like object, which can be used as normal dict, but stores it's data on disk. Both keys and values should be strings.

class foo(znc.Module):
    def OnLoad(self, args, message):
        self.nv['bar'] = 'baz'
        if 'abcde' in self.nv:
            try:
                message.s = self.nv['qwerty']
            except KeyError:
                message.s = self.nv['abcde']
        for k, v = self.nv.items():
            ...

Objects

SWIG distinguish between instances from ZNC and instances created in Python. All instances that are created during python time are garbage collected as soon they leave scope (e.g. at the end of the function or module). To move an instance to the ZNC scope, so it can be used after the lifetime of the function/module, set the special object property .thisown to 0.

For example this is required if you add new user:

 new_user = znc.CUser(username)
 str_err = znc.String()
 if znc.CZNC.Get().AddUser(new_user, str_err):
   new_user.thisown = 0  # new_user won't be garbage collected at the end of the function anymore

Similiar if you want to free an instance (like removing a listener), you should make sure that the memory gets garbage collected

 listener = ...
 if znc.CZNC.Get().DelListener(listener):
   listener.thisown = 1

IRCv3 server-dependent capabilities

Available since ZNC 1.9. (Server-independent caps can be implemented before that, but with the same API as in C++ modules)

While in C++ you'd need to inherit from CCapability, here self.AddServerDependentCapability() accepts two callable objects:

def OnLoad(self, args, ret):
  def server_change(ircnetwork, state):
    self.PutModule('Server changed support: ' + ('true' if state else 'false'))
  def client_change(client, state):
    self.PutModule('Client changed support: ' + ('true' if state else 'false'))
  self.AddServerDependentCapability('testcap', server_change, client_change)
  return True

Web

See WebMods for an overview.

To show your module's page or subpages in the menu, need to define GetWebMenuTitle which should return visual name for the module.

class test(znc.Module):
    def GetWebMenuTitle(self):
        return "Python test module"

CTemplate

// C++
CTemplate& tmpl = ...;
tmpl["name"] = "value";
CTemplate& row = tmpl.AddRow("SomeTable");
row["foo"] = "bar";
# python equivalent (0.206 and below)
tmpl = ...;
tmpl.set("name", "value")
row = tmpl.AddRow("SomeTable")
row.set("foo", "bar")
# python equivalent (since 0.207)
tmpl = ...;
tmpl["name"] = "value"
row = tmpl.AddRow("SomeTable")
row["foo"] = "bar"

Subpages

If you want to have subpages for the module, besides the main page, use helper function znc.CreateWebSubPage. It accepts one required argument - name of the subpage, and several optional arguments:

  • title - text for displaying subpage name. By default it's the same as name.
  • params - dict of parameters which will be used in URL linking to the subpage.
  • admin - set to True if subpage should be accessible only by admins.
def OnLoad(self, args, message):
    self.AddSubPage(znc.CreateWebSubPage('page1'));
    self.AddSubPage(znc.CreateWebSubPage('page2', title='Page N2'))
    self.AddSubPage(znc.CreateWebSubPage('page3', params=dict(var1='value1', var2='value2'), admin=True))
    return True

Timers

Use helper function CreateTimer. It gets following arguments:

  • timer (required) - reference to your Timer class. It should be derived from znc.Timer. You can override 2 methods: RunJob and OnShutdown.
  • interval - Interval between calls, in seconds. Default is 10.
  • cycles - Number of times to run the RunJob function. 0 means infinite. Default is 1.
  • description - Text description of the timer. Default doesn't matter.
  • label - String identifying the timer. If the module already has an active timer with the same label, the new one will not be created. Default is pytimer.
# timertest.py
import znc

class testtimer(znc.Timer):
    def RunJob(self):
        self.GetModule().PutStatus('foo {0}'.format(self.msg))

class timertest(znc.Module):
    def OnModCommand(self, cmd):
        timer = self.CreateTimer(testtimer, interval=4, cycles=1, description='Says "foo bar" after 4 seconds', label='moo')
        timer.msg = 'bar'

You can use methods of C++ class CTimer (like Stop) for your timer.

Sockets

If module needs to know whether ZNC was compiled with IPv6, SSL and c-ares support, you can use special variables, which are True if the feature is supported.

if znc.HaveIPv6:
    ...
if znc.HaveSSL:
    ...
if znc.HaveCAres:
    ...

All sockets are instances of special classes derived from znc.Socket. znc.Socket has all the same methods as Csock, except Connect, Listen and Write. Csock's reference can be found here. To get reference to associated module, use GetModule. Callbacks have different names from ones of Csock, they are described later.

To create socket, use module's method CreateSocket. First argument is reference to your socket class. The function creates socket and calls method Init of it with the rest of arguments. Reference to the new socket is returned.

To connect socket, use method Connect. It gets 2 required arguments - hostname and port, and several optional arguments:

  • timeout - Time in seconds to wait for connection. Default is 60.
  • ssl - Whether to use SSL for connection.
  • bindhost - Local interface to use for the connection.

Returns true value if connection scheduled successfully.

# networkconn.py
import znc
class connsock(znc.Socket):
    def Init(self, line): # line and other arguments, including named arguments can be specified in CreateSocket
        self.Connect('google.com', 80)
        self.EnableReadLine()
        self.Write("{0}\r\n".format(line))
    def OnReadLine(self, line):
        self.GetModule().PutStatus(line) # this puts also \n and \r to status, which is not very good, but this is just an example, so...

class networkconn(znc.Module):
    def OnModCommand(self, cmd):
        sock = self.CreateSocket(connsock, "GET {0} HTTP/1.0\r\n".format(cmd))
# socketconn.py
import znc
class conn(znc.Socket):
    def OnReadLine(self, line):
        self.GetModule().PutStatus(line)

class socketconn(znc.Module):
    def OnModCommand(self, cmd):
        sock = self.CreateSocket(conn)
        sock.Connect('google.com', 443, ssl=True)
        sock.EnableReadLine()
        sock.Write("GET {0} HTTP/1.0\r\n\r\n".format(cmd))

To create listening socket, use method Listen. It gets following optional named arguments:

  • port - Port number to listen on. If not presented, random port is choosed.
  • bindhost - Interface to listen on. If not presented, socket will listen on all interfaces.
  • addrtype - Chooses protocol family. Possible values are 'all', 'ipv4' and 'ipv6'. Default is all.
  • ssl - Whether to use SSL for incoming connections.
  • maxconns - Maximum number of connections. Default is SOMAXCONN.
  • timeout - time in seconds, for timeout.

Returns 0 on error and port number on success.

# listmodule.py
import znc

class accepted(znc.Socket):
    def Init(self, host, port):
        self.Write("Hello, {0}:{1}!\n".format(host, port))
    def OnReadData(self, data):
        self.WriteBytes(data) # echo back everything

class listensock(znc.Socket):
    def OnAccepted(self, host, port):
        return self.GetModule().CreateSocket(accepted, host, port)

class listmodule(znc.Module):
    def OnLoad(self, args, message):
        sock = self.CreateSocket(listensock)
        port = sock.Listen(ssl=True, addrtype='ipv6');
        if port > 0:
            message.s = "Listening on all IPv6 interfaces on port {0} using SSL".format(port)
        return True

Use Write to write strings, and WriteBytes to write binary data.

Sockets can override following callbacks:

  • Init - is called from CreateSocket, first argument is reference to socket, the rest is from arguments to CreateSocket.
  • OnConnected
  • OnDisconnected
  • OnTimeout
  • OnConnectionRefused
  • OnReadData - gets bytes as second argument
  • OnReadLine - is called for every new line from socket. The line, including ending \n (or \r\n) is in argument. It's called only if you enabled this feature for the socket.
  • OnAccepted - is called for listening socket for every new connection. Arguments are hostname and port of remote end. The callback should return None if you don't need the connection, or reference to new socket, which will be used for this connection.
  • OnShutdown - destructor of the socket.

If callback On* raises an exception, the socket is closed, but if you want to close socket, use method Close instead. If Init raises an exception, behavior is undefined.

Getting ZNC version

If you just want to show ZNC version to humans, usually just znc.CZNC.GetTag() is good.

znc.CZNC.GetTag() # Returns, for example, 'ZNC 0.097 - http://znc.sourceforge.net'

Check ZNC C++ documentation for details.

For getting ZNC version, you can use read following variables:

znc.Version # For example, number 0.097
znc.VersionMajor # 0
znc.VersionMinor # 97
znc.VersionExtra # build-specific string