Midterm Summary

This midterm was full of surprises and challenges for me. During it I was working on two features that I should add to Splash:

• splash:with_timeout
• Element class

with_timeout

Several return values

splash:with_timeout is the first API that I’d implement during this summer. Originally, it was planned to be a simple flag for splash:go but become something bigger and more practical. More about it you can read in my previous post.

The other challenge that occurred during the development is the ability to return several values from the passed function. Let’s look on this example:

function main(spash)
    local ok, result1, result2, result3 = splash:with_timeout(function()
        return 1, 2, 3
    end, 0.1)

    return result1, result2, result3
end

Without any change this script will return [[1, 2, 3], None, None]. It happens because the result values of the passed \ to splash:with_timeout function are converted to Python’s list and then it passed back to Lua, where it becomes a table.

To fix that I wrote a Lua wrapper which packs the result values of the function and then unpacks it back to ensure that it looks like it was originally. Before doing this task I didn’t have so much experience with Lua and how Lua to Python and Python to Lua works in Splash. That’s why I spent more than a week to do it.

Docs

Another thing in which I didn’t have much experience is a documentation writing. In Splash (and any other open-source project) you should write a comprehensive documentation about your implemented API, with examples and explanations of how it works.

I want to mention one aspect of Splash. It’s Lua scripting engine is implemented using custom written event loop. And because of that splash:with_timeout may not stop the running function if the timeout expires (you can do some blocking operation which will stop the entire event loop). And that aspect should be written in docs and explained as simple as possible, which is not very easy.

Element

Element class is supposed to be a wrapper for a DOM element with utility methods. Last week I’ve started working on it. One of the interesting parts of this API is how DOM element is stored in Python and Lua. I’ve decided to do in the following way:

1. When the JS window object of the page is created I assign to it a special object for storing DOM elements.
2. Element class is created using CSS selector. It passed to JS and DOM element is retrieved using document.querySelector and stored with UUID in the storage which was created in step 1.
3. That UUID is passed to Python which is assigned to element object property.
4. The further operations are performed using that UUID.

I only implemented the steps which I described below and I’m going to implement Lua interface for performing those steps.

Thank you for reading. See you next time 😉