Posts Tagged ‘ffmpeg’

June 2018 Development Update

Tuesday, July 31st, 2018 by fireclaw

Due to the vacation period, this post is somewhat delayed, but the wait is finally at an end. Here is the new update with a selection of the developments in June.

OpenGL on macOS

At the Apple WWDC in June of this year, Apple formally announced the deprecation of the OpenGL graphics API, in favor of an Apple-only graphics API called Metal. This move puzzled many as a lot of software is relying on OpenGL for macOS support, including Panda3D, and investing significant resources into developing a whole new rendering back-end to support just a relatively small segment of the market is hard to justify.

While it seems likely that—despite the deprecation notice—Apple will continue to include OpenGL support as a part of macOS, we will need to start looking at another approach for maintaining first-class support for high-end graphics on macOS. It would be nice if there were one next-gen graphics API that would be well-supported on all platforms going forward.

Enter MoltenVK. This is an implementation of the cross-platform Vulkan API for macOS and iOS, implemented as a wrapper layer on top of Apple’s Metal API. It has recently been open sourced by the Khronos group, in an effort to make Vulkan available on every operating system. Despite being a wrapper layer, it was still found by Valve to have increased performance benefits over plain OpenGL. This will let us focus our efforts on implementing Vulkan and thereby support all platforms.

Accordingly, we have increased the priority towards developing a Vulkan renderer, and it has made several strides forward in the past months. It is still not quite able to render more than the simplest of sample programs, however. We will keep you updated as developments unfold.

Mouselook smoothness improved

In order to implement camera movement using the mouse, many applications use move_pointer to reset the cursor to the center of the window every frame. However, when the frame rate gets low, the mouse movement can become choppy. This is usually only an issue occurring on Windows as other platforms support the M_relative mouse mode, which obviates the need to reset the mouse cursor. For all those who can’t or don’t use this mode, a significant improvement in smoothing this movement has now been worked out.
In previous versions, small movements that occurred between the last iteration of the event loop and the call to move_pointer could have been ignored, causing the mouselook to become choppy. This has been fixed by changing the get_pointer() method to always obtain the latest mouse cursor position from the operating system, resulting in more accurate mouse tracking. In the future, we will further improve on this by emulating relative mouse mode on Windows and perhaps even adding a new interface for handling mouselook.

Grayscale and grayscale-alpha support with FFmpeg

Panda3D’s implementation of FFmpeg now gained the ability to load videos that only contain grayscale colours more efficiently, greatly reducing the memory usage compared to the previous approach of converting all videos to full RGB. This is also extended to videos that have a grayscale channel as well as an alpha channel for transparency. Do note, however, that not all video formats support grayscale pixel formats.

Removing ‘using namespace std’ from Panda3D headers

C++ users, take note: besides the changes to add better C++11 support, we have also eliminated the bad practice of having using namespace std; in the public Panda3D headers. This prevents the entire std namespace from being pulled in and causing namespace clashes. If your codebase relied on being able to use standard types without an explicit std:: prefix, you may need to add using namespace std to your own headers. It would be even better to fully qualify your references to standard types and functions with an std:: prefix or—where appropriate—pull in specific types with individual using std::x; statements.

Bullet Vehicle chassis-to-wheel synchronisation

Using Bullet’s BulletVehicle class to simulate car physics had a problem with syncing the wheels to the chassis if the wheels were parented to the the vehicle’s RigidBodyNode. The wheel models would always be one frame ahead of the chassis model, resulting in visible artifacts as the wheels would appear to move away from the chassis. Especially if the vehicle accelerates to a significant speed, the wheels may appear to drift quite far from the vehicle. This bug was fixed thanks to a contributor.

March 2018 Development Update

Friday, April 20th, 2018 by fireclaw

Winter is over and it’s time for a spring-cleaning. Many issues have been fixed this month and we’ve also completely refreshed the forum system. Also, changes have been made towards cleaning up the codebase and removing obsolete code. Keep an eye open for the awesome things our developers have in store for later this year!

Forum

The forums got a complete overhaul and has been moved over to Discourse, a 100% free and open source discussion board with plenty of neat features and a modern look. It has become quite a popular system of late, so it may appear familiar to some. The new software is considerably better at preventing spam and the increased mobile usability, code highlighting support and better notifications are welcome features as well. However, due to the conversion to Markdown, some formatting in old forum posts may not have been converted correctly.
If you have any trouble or concerns using the new forum, let us know! We’ll do our best to make the transition as smooth as possible.

Adjacency Geometry

Silhoutte rendering using a geometry shader. Model by Christophe Seux.

Silhoutte rendering using a geometry shader (click to enlarge). Model by Christophe Seux.

Support has been added to the OpenGL renderer for adjacency information. This is primarily useful for geometry shaders that need to access information about vertices connected to the primitive that is being processed, which is useful when implementing effects such as silhouette rendering (as seen in the screenshot to the right). Panda3D can automatically generate adjacency information for existing meshes, but can also take precomputed adjacency information.

Logging in deploy-ng

A new addition to deploy-ng which could help with debugging deployed apps—especially on Windows—is the possibility to provide a path to a log file to which a packaged app will automatically write all of its output. As GUI applications on Windows can’t write to the console, this new way of logging can be used to easily reroute the output to a log file without having to write an extra logging mechanism in your application.
To enable this feature for your application, you simply have to set the log_filename variable in your setup.py script. For an example, take a look at the setup script in the asteroids example in the deploy-ng branch. By default the log file will be rewritten every time, but with a simple switch of the log_append variable, you can preserve the previous content.

Dropping support for MSVC 2010

Since we are continuing to take greater advantage of C++11 features in the Panda3D codebase, we can no longer support the Microsoft Visual C++ 2010 compiler. The 2015 version has been made the new default for building Panda3D from source on Windows systems, and we intend to cease supporting 2010 very soon. If this is causing issues for you or your team, please make your voice heard as soon as possible on the corresponding GitHub issue!

Dropping Mac OS X 10.6 “Snow Leopard” support

Furthermore, we also intend to stop supporting Mac OS X 10.6 “Snow Leopard” due to the limits it imposes on how much of the C++11 standard can be used in the Panda3D codebase. However, if your application is still targeting this version of macOS, please give your opinion on the corresponding GitHub issue.

Video4Linux

Panda3D now supports the grayscale pixel format on Linux which is used for example by some infrared (IR) cameras. Additionally, the reading of video frames from the camera is now done asynchronously, meaning that the framerate of the application is no longer locked to the framerate of the camera in Augmented Reality applications. The previous behaviour can be restored by enabling the v4l-blocking configuration variable.

Better Python 3 Support

Support for Python 3 has been enhanced by replacing usage of the C++ string type with vector_uchar where it is used to refer to binary data. Since all strings are seen as UTF-8 now, this avoids exceptions which were raised due to confusion about whether buffers contained UTF-8 strings or binary data. Convenience functions have been added for efficiently dealing with the new way of representing binary buffers.

Media decoding and playing

CFSworks, who regularly contributes to the project, has done some work on various parts of the audio system, among many other changes. Firstly, he implemented workarounds for some rare OpenAL implementation bugs which are particularly prevalent on the Apple implementation. He also helped to clean up use of deprecated APIs in the FFMpeg integration and implemented features available to newer versions of this library. We are very grateful to him for his continued contributions.

A look behind the curtains

Monday, November 20th, 2017 by fireclaw

Much has happened in Panda3D development for the upcoming 1.10 version. To bring you up-to-date with the latest developments, we will summarize some of the new changes here. Also, to further keep you informed about new and upcoming features, we’ll start a regular blog post series highlighting new developments.

Aside from a lot of optimization changes to improve various parts of Panda’s performance, as well as numerous bugfixes to improve stability and compatibility, there were some larger changes as well.

Python support

The first thing we’d like to highlight is the ability for Python users to install Panda3D via the pip package manager. No more fiddling with platform dependent installers—it takes only a single command to install the right version of Panda3D for your platform and Python version:

pip install panda3d

As a bonus feature, this allows you to install Panda into a virtualenv environment, allowing you to try out the latest development version in isolation without fear of contaminating your existing setup.

Furthermore, Panda3D has been updated to be compatible with the latest Python 3 versions. This includes interoperability with the pathlib module and the Python 3.6 path protocol, as well as fixes for the upcoming Python 3.7.

The Shader Generator

If you are using the shader generator in your application, you may significantly benefit from upgrading to 1.10. It has been overhauled to address a major performance concern for applications with complex scenes containing a large amount of render states, which could cause lag due to an excessive amount of shaders being generated.

Some new features have been added as well, such as support for hardware skinning and multiple normal maps.

Text rendering updates

The text rendering subsystem has been improved significantly. Panda’s text assembler used to perform well mainly for smaller texts, whereas frequently updating large blocks of text could cause considerable lag. But the improved text assembler code is up to 75 times as fast, making assembling large swaths of text a non-issue.

A comparison with HarfBuzz disabled and enabled. Of note is the spacing between the A and V, the "fi" ligature. The Arabic text renders like a mess without HarfBuzz.

A comparison with HarfBuzz disabled and enabled. Of note is the spacing between the A and V, the “fi” ligature. The Arabic text doesn’t render correctly at all without HarfBuzz.

Furthermore, the HarfBuzz library can now be utilized to implement text shaping, which not only enables support for ligatures and correct kerning but also allows us to better support languages with more complex shaping requirements, such as Arabic. This includes support for right-to-left text rendering, with automatic language detection enabled by default. Although bidirectional text is not yet fully supported, you can explicitly switch or re-detect direction for specific text segments using embedded TextProperties.

If Panda3D has been compiled with HarfBuzz support, it can be enabled using the text-use-harfbuzz variable. Otherwise, more basic kerning support can be enabled using text-kerning true, although many fonts will only kern correctly with HarfBuzz enabled.

Media playback

Panda3D now directly supports the Opus audio codec, a high-quality open standard designed to efficiently encode both speech and other audio. This is implemented via the opusfile library, so that it doesn’t require pulling in the heavier and more restrictively licensed FFmpeg libraries.

The FFmpeg plug-in now also supports loading video files with an embedded alpha channel, such as is possible with WebM files encoded with the VP8 codec. However, FFmpeg offers both a preferred native implementation and a decoder based on libvpx. The default is the native implementation, so if you wish to play VP8 videos with alpha channel, you should set the ffmpeg-prefer-libvpx configuration variable to true, to force FFmpeg to use the libvpx implementation.

 

We’d also like to highlight ongoing work outside the main Panda3D development branch. These things have been developed for Panda3D and will be merged into the main branch when they have reached maturity. But until then, they can be checked out from their respective branches on GitHub.

Deployment

First off, significant progress has been made on a new deployment system thanks to invaluable contributions by the community. The project is tentatively named “deploy-ng” and intends to make it easier more reliable to package and distribute your finished application, and as such it stands to replace the current deployment system entirely.

This new deployment system builds upon the existing Python setuptools, adding an extra plug-in to easily package your Panda3D applications. It already is quite usable, but still needs some love and testing until it’s production ready.

Graphics back-ends

A significant amount of work has been done on the effort to support two new graphics back-ends. The first of these is the WebGL back-end, happening on the webgl-port branch. This allows us to run Panda3D applications in the browser without requiring the use of a browser plug-in. The bulk of the work on the renderer itself has already been done, but there remains work to be done to make it easier to package up a Panda application for the web. Check out the proof-of-concept demos or the online editor demo if you’re curious about the possibilities.

On the vulkan branch, a prototype renderer for the new Vulkan graphics API has materialized as well. Like OpenGL, Vulkan is a cross-platform open graphics standard developed by Khronos. Unlike OpenGL, however, Vulkan offers a more low-level interface to the underlying graphics hardware, enabling a reduction in driver overhead for CPU-bound applications. Before you get too excited though, it’s not yet capable of running much more than a few of the sample programs. There is a lot more work to be done before it will reach feature-parity with or performance benefits over the OpenGL renderer, and it is unlikely to be a priority for the next release.

glTF 2.0

Behind the curtains there also is work going on to support glTF 2.0. This is a new JSON-based model transmission format recently standardized by the Khronos Group, the consortium that is also responsible for OpenGL, and plug-ins are already available to export it from various content creation tools. Importantly, glTF 2.0 defines a modern standard for physically-based materials, and as such is considered a milestone in the development of a physically-based rendering pipeline in Panda3D.

Input devices

Gamepad support is something that many in the community have been asking about for a long time. The input framework is receiving a significant overhaul to allow us to support game controllers, while also laying the groundwork for exposing commercial virtual reality hardware using a straightforward API. This work is happening on the input-overhaul branch and will be merged into the master branch soon.

 

That’s all for now, but keep an eye open for upcoming blog posts with all new and interesting updates in the coming months. In the meantime we encourage you to try the latest version for yourself and let us know how it works for you.