May 2018 Development Update

With the work on the new input system and the new deployment system coming to a close, it is high time we shift gears and focus our efforts on bundling all this into a shiny new release. So with an eye toward a final release of Panda3D 1.10, most of the work in May has centered around improving the engine’s stability and cleaning up the codebase.

As such, many bugs and regressions have been fixed that are too numerous to name. I’m particularly proud to declare the multithreaded render pipeline significantly more stable than it was in 1.9. We have also begun to make better use of compiler warnings and code-checking tools. This has led us find bugs in the code that we did not even know existed!

We announced two months ago that we were switching the minimum version of the Microsoft Visual C++ compiler from 2010 to 2015. No objections to this have come in, so this move has been fully implemented in the past month. This has cleared the way for us to make use of C++11 to the fullest extent, allowing us to write more robust code and spend less of our time writing compiler-specific code or maintaining our own threading library, which ultimately results in a better engine for you.

Thinking ahead

Behind the scenes, many design discussions have been taking place regarding our plans for the Panda3D release that will follow 1.10. In particular, I’d like to highlight a proposed new abstraction for describing multi-pass rendering that has begun to take shape.

Multi-pass rendering is a technique to render a scene in multiple ways before compositing it back together into a single rendered image. The current way to do this in Panda3D hinges on the idea of a “graphics buffer” being similar to a regular on-screen window, except of course that it does not appear on screen. At the time this feature was added, this matched the abstractions of the underlying graphics APIs quite well. However, it is overly cumbersome to set up for some of the most common use cases, such as adding a simple post-processing effect to the final rendered image. More recent additions like FilterManager and the RenderPipeline’s RenderTarget system have made this much easier, but these are high-level abstractions that simply wrap around the same underlying low-level C++ API, which still does not have an ideal level of control over the rendering pipeline.

That last point is particularly relevant in our efforts to provide the most optimal level of support for Oculus Rift and for the Vulkan rendering API. For reasons that go beyond the scope of this post, implementing these in the most optimal fashion will require Panda3D to have more complete knowledge of how all the graphics buffers in the application fit together to produce the final render result, which the current API makes difficult.

To remedy this, the proposed approach is to let the application simply describe all the rendering passes up-front in a high-level manner. You would graph out how the scene is rendered by connecting the inputs and outputs of all the filters and shaders that should affect it, similar to Blender’s compositing nodes. You would no longer need to worry about setting up all the low-level buffers, attachments, cameras and display regions. This would all be handled under the hood, enabling Panda3D to optimize the setup to make better use of hardware resources. We could even create a file format to allow storing such a “render blueprint” in a file, so something like loading and enabling a deferred rendering pipeline would be possible in only a few lines of code!

This is still in the early design stages, so we will talk about these things in more detail as we continue to iron out the design. If you have ideas of your own to contribute, please feel free to share them with us!

Helping out

In the meantime, we will continue to work towards a final release of 1.10. And this is the time when you can shine! If you wish to help, you are encouraged to check out a development build of Panda3D from the download page (or installed via our custom pip index) and try it with your projects. If you encounter an issue, please go to the issue tracker on GitHub and let us know!


April 2018 Development Update

Welcome back to our monthly developer news. This month we don’t have much interesting to offer on the current developments. Most time has been spent to fix bugs and iron out existing functionality. So we decided to provide you with some outlook on which future developments you can expect to see next.

New app framework

Behind the scenes, much discussion has taken place regarding future developments of Panda3D.  In this post, we will discuss one of the plans on the table, namely to redesign the high-level application layer of the Panda3D library.

The current design of the ShowBase class as a monolithic singleton makes it easy to get started with prototyping a game in Panda, but it does have some downsides. It is cluttered with methods and variables that may not be needed by every application. It also makes running two instances of the application within the same process harder, and because it is implemented in Python, it is not available to C++ developers.

Our intent is to introduce a replacement for ShowBase that not only resolves these issues, but also encourages best practices and makes it easier for developers to write games for non-desktop platforms, such as mobile, web and VR.

For example, the ShowBase design gives the application developer complete agency over the lifetime of the application.  This is a great fit for traditional desktop applications, but not so great for mobile apps, where the operating system needs to be able to suspend and resume applications at any time in order to free up resources when needed.  Doing this today in Panda3D requires developers having to specifically detect these states, whereas we would ideally make this an intuitive part of the game structure.

Another example is virtual reality, where there is not really a meaningful “window” in the traditional sense, but the content is rather projected into a virtual 3D environment, with any GUI being superimposed into the scene in layers.  We would like to represent these concepts in a way that makes it easy for developers to develop applications for both VR and desktop, while giving them full control over the additional possibilities that virtual reality offers.

While the intent is not to remove control that game developers might need, we want to make a framework that encourages best practices that work well for all platforms.  We will follow up on this in the future with more concrete plans, but rest assured that ShowBase will remain available for the foreseeable future for any applications that rely on it.

CMake

CMake has in recent years rapidly become the de facto standard build system for open-source projects. As we have reported on before, we are working on replacing Panda’s own build system “makepanda” with it. This work is still underway on the cmake branch in GitHub.

For Panda3D developers, the integration of CMake in IDEs and other tools is far better than that of makepanda. This also allows us to better integrate the testing framework and code checking tools. Using CMake also simplifies building and directly running a built Panda3D instance (without installing) from the build directory using the rpath feature on Linux.
Another benefit is the simplified cross-compilation and exotic compiler support for mobile or ARM-based systems. In the future there might be support for CMake’s “exported targets” feature, meaning C++ users of Panda3D would not have to configure their libraries/includes/search path if they’re using CMake as well. Another exciting benefit is that CMake can lower the build times by roughly 30% dependent on the system and setup.
A little side effect of using CMake instead of makepanda is that we have less code to maintain and we get the benefits of the CMake community’s development efforts as a whole. That also means we get to focus our resources exclusively on making the engine itself awesome as we’re not distracted with the build tools.

The CMake branch is currently in a good enough state where it works great on Linux/BSD, and macOS support isn’t far behind. Windows support is slightly trickier and needs some more work before it gets up to that level of support. Mobile platforms are not yet supported.

If you run Linux or macOS feel free to grab the branch and try it for yourself. Whenever you find any bugs, just fill a report on the GitHub issue tracker or ask in the forum or on the IRC channel.

Ubuntu 18.04 “Bionic Beaver”

The most recent LTS version of Ubuntu, 18.04 has been released a few weeks ago with recent software packages. Panda3D has been successfully built on this version and updated install packages are available for the 1.10 development builds.


March 2018 Development Update

Winter is over and it’s time for a spring-cleaning. Many issues have been fixed this month and we’ve also completely refreshed the forum system. Also, changes have been made towards cleaning up the codebase and removing obsolete code. Keep an eye open for the awesome things our developers have in store for later this year!

Forum

The forums got a complete overhaul and has been moved over to Discourse, a 100% free and open source discussion board with plenty of neat features and a modern look. It has become quite a popular system of late, so it may appear familiar to some. The new software is considerably better at preventing spam and the increased mobile usability, code highlighting support and better notifications are welcome features as well. However, due to the conversion to Markdown, some formatting in old forum posts may not have been converted correctly.
If you have any trouble or concerns using the new forum, let us know! We’ll do our best to make the transition as smooth as possible.

Adjacency Geometry

Silhoutte rendering using a geometry shader. Model by Christophe Seux.

Silhoutte rendering using a geometry shader (click to enlarge). Model by Christophe Seux.

Support has been added to the OpenGL renderer for adjacency information. This is primarily useful for geometry shaders that need to access information about vertices connected to the primitive that is being processed, which is useful when implementing effects such as silhouette rendering (as seen in the screenshot to the right). Panda3D can automatically generate adjacency information for existing meshes, but can also take precomputed adjacency information.

Logging in deploy-ng

A new addition to deploy-ng which could help with debugging deployed apps—especially on Windows—is the possibility to provide a path to a log file to which a packaged app will automatically write all of its output. As GUI applications on Windows can’t write to the console, this new way of logging can be used to easily reroute the output to a log file without having to write an extra logging mechanism in your application.
To enable this feature for your application, you simply have to set the log_filename variable in your setup.py script. For an example, take a look at the setup script in the asteroids example in the deploy-ng branch. By default the log file will be rewritten every time, but with a simple switch of the log_append variable, you can preserve the previous content.

Dropping support for MSVC 2010

Since we are continuing to take greater advantage of C++11 features in the Panda3D codebase, we can no longer support the Microsoft Visual C++ 2010 compiler. The 2015 version has been made the new default for building Panda3D from source on Windows systems, and we intend to cease supporting 2010 very soon. If this is causing issues for you or your team, please make your voice heard as soon as possible on the corresponding GitHub issue!

Dropping Mac OS X 10.6 “Snow Leopard” support

Furthermore, we also intend to stop supporting Mac OS X 10.6 “Snow Leopard” due to the limits it imposes on how much of the C++11 standard can be used in the Panda3D codebase. However, if your application is still targeting this version of macOS, please give your opinion on the corresponding GitHub issue.

Video4Linux

Panda3D now supports the grayscale pixel format on Linux which is used for example by some infrared (IR) cameras. Additionally, the reading of video frames from the camera is now done asynchronously, meaning that the framerate of the application is no longer locked to the framerate of the camera in Augmented Reality applications. The previous behaviour can be restored by enabling the v4l-blocking configuration variable.

Better Python 3 Support

Support for Python 3 has been enhanced by replacing usage of the C++ string type with vector_uchar where it is used to refer to binary data. Since all strings are seen as UTF-8 now, this avoids exceptions which were raised due to confusion about whether buffers contained UTF-8 strings or binary data. Convenience functions have been added for efficiently dealing with the new way of representing binary buffers.

Media decoding and playing

CFSworks, who regularly contributes to the project, has done some work on various parts of the audio system, among many other changes. Firstly, he implemented workarounds for some rare OpenAL implementation bugs which are particularly prevalent on the Apple implementation. He also helped to clean up use of deprecated APIs in the FFMpeg integration and implemented features available to newer versions of this library. We are very grateful to him for his continued contributions.


February 2018 Development Update

We again bring you some of the highlights of the new developments in the past month. This is however not an exhaustive list of changes, which can be obtained from the commit logs in the GitHub repository.

Android developments

We’re happy to announce that the Android port has made great strides forward in the past weeks. Most importantly, there is now a complete Python interpreter added to the Panda3D app, so that Python applications can be run directly from the Android device. Many of the sample programs now run as well as they do on the desktop.

It is now even possible to compile Panda3D entirely on an Android device, using an app that provides a Linux-like terminal environment. One free and open-source example is termux, which provides a package manager that gives access to an impressive suite of software, including the compilers needed to compile Panda3D.

Furthermore, using a bash script that we provide, it is also possible to launch Panda3D from termux and pipe the output from the interpreter app back to the terminal, so that you can develop Panda apps on Android the same way as you would on the desktop.

Eventually, we intend to make it possible to use the new deploy-ng system to produce an Android package from any operating system, making it easy to deploy your application to both desktop and mobile platforms.

In a separate effort, a commercial port of Panda3D to Android has been released on the app store, called Cub3D. This is an impressively complete development environment for Panda3D apps, and can be used today to develop and run Panda apps on Android. (This software is developed by a third-party company that is not affiliated with the Panda3D team.)

Roaming Ralph running on an Android tablet

Roaming Ralph running on an Android tablet

Ball in Maze running on an Android watch with Cub3D

Ball in Maze running on an Android watch with Cub3D



CMake improvements

Behind the scenes, we have been working on adding support for the CMake build system on the cmake branch. CMake is a popular cross-platform build system that is rapidly becoming the de facto standard for open-source projects, and it is eventually intended to replace our own makepanda build script. This will make it easier to build Panda from source and to customize the build process. Last month, the CMake port saw some updates and improvements, bringing us closer to this goal.

Bullet thread safety

Users of the Bullet physics engine were previously encountering crashes when using Bullet in conjunction with the multithreaded render pipeline. This is caused by the fact that the Bullet library is not thread safe. To remedy this, we have added locking in the Panda3D wrapper for Bullet that protects the Bullet library from access by multiple threads, which makes it safe to use the Bullet integration in a multithreaded program.


January 2018 Development Update

The new year has brought with it new developments to the Panda3D engine, some of which we would like to present to you today. This is however by no means a comprehensive listing of the improvements we’re working on. Stay tuned for more posts, as we’ve got some exciting plans for 2018 ahead!

RenderPipeline light system

The light manager of Tobias Springer’s excellent RenderPipeline project has made its way into the Panda3D codebase. This is a light system designed to be used in conjunction with the GPU light culling and deferred rendering methods provided by the RenderPipeline, and is implemented in C++ for optimal performance. Now that it is included with Panda3D, it is even easier to use the RenderPipeline in your projects as it is no longer necessary to compile any C++ modules to do so—you now simply put the Python module into your project and follow the usual steps from there on out.

This feature is mainly useful from a RenderPipeline setup, but we are continuing to work on bringing the built-in lighting system more closely in line with the RenderPipeline lights. Examples of this are additional light types such as sphere and rectangle area lights and the possibility to set a light’s color temperature.

Input device support

The input-overhaul branch has received plenty of changes again and is day by day getting closer to a state where it can be merged into the master branch. The latest improvements include more devices being supported as well as overall improved handling and mapping of devices’ buttons and axes, such as for joysticks. For Windows users, there is also a new input manager available based on the Windows raw input system, which is used for devices that are not supported under the existing XInput implementation. Panda3D automatically chooses the right implementation to use for a device, so this all happens seamlessly to the developer.

Support for 3D mice has also been added. This is a class of devices that allow movement in six degrees of freedom (thrice as many as a regular mouse), and is particularly used by 3D artists for intuitively navigating a camera around a model or through a scene. This may be of particular use for the various CAD programs that are built around Panda3D.

Android developments

Android support is actively being worked on and great strides have already been made in this area. Stay tuned for the next post, in which we will have some exciting announcements to make on this front!


December 2017 Development Update

Hello everyone to our heavily delayed December post. Even though we’re quite late with this one due to lack of time, we wish you all a happy new year! Much has happened during the winter holidays, so read on to see what’s new.

What happened in the last month

The work on the input overhaul branch is almost complete and needs some more polish to finalize the API before we will merge it into the development trunk. In addition, we started to add a mapping table for known devices to have them work as expected. For other devices, the mapping is provided by the device driver with the help of some heuristics to detect the device type. Currently the input overhaul is still in heavy development and API changes will occur, though for those who are interested in testing it, sample applications are available and some manual entries with more or less accurate instructions have been created and will be finalized as soon as the API is stable.

Some long-standing bugs with the multithreaded pipeline were finally resolved. These issues caused deadlocks that occurred whenever geometry data was being manipulated on another thread, or when shadows were enabled using the automatic shader generator; as such, they were a significant barrier that prevented the multithreaded pipeline from being useful for some applications. However, more testing is needed before we can be completely confident that all the threading issues are resolved.

On macOS, it is now possible to get an offscreen rendering buffer without opening a window. This lets you render to a buffer on a headless mac server, which can be useful for plenty of things. Aside from scientific simulations where no immediate output is necessary or even desirable, another example is to send a frame rendered by Panda3D over a socket or network to display it elsewhere. This technique is used in the BlenderPanda project to render a Panda3D frame into a Blender viewport and thereby get a live display of how a model will look when used with the engine.

Looking into the crystal ball

In the coming months some of the newly developed features (input-overhaul, deploy-ng) will be polished off and merged into the master branch of panda3d. More work is also planned on the introduction of a new physically-based material model as well as support for the glTF 2.0 format. Stay tuned for more updates!


November 2017 Development Update

Hello and welcome to our first official informative post in this series of Panda3D development blog posts. We’ll showcase newly added features as well as behind-the-scenes information and other changes in November, 2017.

Note that this is just an overview highlighting a few of the bigger features that were added this month. It is by no means a comprehensive list of all the features and bugfixes that were added. For a full list of changes, check out the git commit log.

What happened in the last month

Unit testing

Panda3D got initial test coverage utilizing the mature and full-featured PyTest toolset. This means that core features of the engine will always be tested for their intended functionality and bugs in those places will be caught earlier, helping to keep the engine in an overall good and stable state. Currently there is basic test coverage for NodePath, serializability of Bullet shapes, various interrogate features and some display configurations, but we plan to add more in the future.

Coroutines and async/await

The most exciting change this month is perhaps the support for coroutines and async/await. This is a new syntax introduced in Python 3.5 to simplify asynchronous programming. It can be hard to understand at first, but the possibilities are quite interesting.

Simply put, a “coroutine” is a function that can be temporarily suspended, pending an asynchronous operation, and continued at a later time. For example, if you use a coroutine to set up your scene, you can temporarily suspend it while a model is being loaded asynchronously, so that other operations can continue without blocking the rendering loop.

Asynchronous operations were already possible in Panda using callbacks, but these get messy quite quickly when dealing with multiple operations. The async/await syntax introduced in Python 3.5 makes it significantly easier to write code that involves a series of asynchronous operations.

Here is a side-by-side example of loading a scene using the new approach. (Use of classes is omitted for brevity.) Note that coroutines need to be added to the task manager to make use of them.

# With coroutines

async def loadScene():
    text = OnscreenText("Loading…")

    # Suspend the coroutine while waiting for it to load
    scene = await loader.loadModel("scene"blocking=False)
    scene.reparentTo(render)

    text.hide()

taskMgr.add(loadScene())
# With callbacks

def loadScene():
    global text
    text = OnscreenText("Loading…")
    loader.loadModel("scene"callback=loadSceneCallback)

def loadSceneCallback(scene):
    scene.reparentTo(render)
    text.hide()

loadScene()

The way this works is that loadModel, when passing blocking=False, will schedule the load operation to happen in another thread, and immediately return a ‘future’ object. This is a special object (called a ‘promise’ in some languages) that represents an operation that is not yet complete, but will later return a result. The ‘await’ keyword is what suspends the operation of the coroutine until the future is complete. At a later time, the task manager notices that the future is done, and will resume execution of the loadScene function.

The ‘await’ keyword isn’t only supported when loading models. Any task has been made ‘awaitable’, meaning you can suspend a task waiting for another task to complete. You can also ‘await’ various other operations that can occur in the background. At present, that includes:

  • any task or asynchronous loader request
  • messenger.future('event') (waits for the given event to be thrown)
  • interval.start() (waits for the interval to complete)
  • win.triggerCopy() (waits for the copy operation to be complete)
  • texture.prepare() and shader.prepare() (waits for the resource to be prepared)

Coroutines need to be added to the task manager, but they can also be used as event handlers. Here is an example of an asynchronous event handler that uses ‘await’ to time how long the spacebar was held, without needing to store external state:

async def on_press_space():
    # Record the time at which the space key was pressed.
    begin = time.time()

    # Suspend the method until the space-up event is detected.
    await messenger.future("space-up")

    # Now measure how much time it took.
    duration = time.time() - begin
    print("spacebar was held for {} seconds".format(duration))

base.accept("space", on_press_space)

This is still an experimental feature, and we’d love to get your feedback on how we can further improve support for asynchronous programming in Panda3D.

Bullet physics

With the help of the community, all Bullet Physics engine shapes supported by Panda3D have now been made serializable. This means all Bullet structures can now be written to a .bam file, paving the way for future model conversion tools to implement support for Bullet physics.

Interrogate property interfaces

It has been a long-requested feature to enable property interfaces in the Python bindings, as a way to access members instead of the traditional getters and setters. This has been developed further by refining the support for sequence and mapping properties. For example, instead of the get_tag() and set_tag() methods, NodePath now also exposes its tags through a tags property that behaves like a Python dictionary, implementing many standard dictionary methods such as update() and keys(). The same applies to properties that expose collections, providing a predictable, standard and aesthetic interface to manipulate the API.

Released games section

In this section we’ll give you an overview of released and updated game titles that have been written with Panda3D

New game releases

The Traveling Fox

Updated games

Kitsunetsuki
YORG