As someone who does a lot of custom video processing, there is one crucial functionality I find lacking in almost all programming languages I come across:
  1. Loading a video + audiotrack (e.g. Testvid.mkv) with 1 line of code
  2. Having FAST read access to both the video frame pixels and audio samples
  3. Being able to use the CPU/GPU (GLSL shaders for example) to process video pixels and audio sample data efficiently
  4. Again, 1 code line easy video + audio save function with CODEC/Compression options

We live in age DOMINATED by audiovisual content.

Yet no serious programming language seems to come with good built in Audio/Video file Read/Process/Write support.

I was thinking that Nim could be an exception to this.

Imagine Nim being able to read/write common audio/video files with 1 line of code without having to call 3rd party libraries or .dlls.

There are tens of thousands of real-world applications that require some form of custom realtime or offline image/audio/video processing.

If Nim runs as fast as native C and had audio/video read/write capabilities baked into the core language, you could be looking at the new DARLING of the image/audio/video processing world.

This is my suggestion to any Nim developers reading this forum - please add audiovisual capabilities into the core language itself.

Everyone faced with having to code custom image/audio/video processing algos would thank you for this functionality.

Currently, many people run to less-than-ideal solutions like Matlab, Videolab, Processing to do audiovisual processing with code.

A fast compiled language like Nim with built-in read/write support for multimedia files could attract a tremendous number of people to Nim.

Thanks For Reading,

VideoBuddha

2017-01-21 21:47:42

Not quite what you are proposing, but there is an opencv nimble package (only a partial wrapper, IIRC).

I wonder how realistic implementing this would be given the Copyright issues with various codecs.

2017-01-22 10:38:04

I don't agree. This is an area that is great for you, and would be very interesting for a lot of other people. But not generally. Just think about video codec developers. How would a video codec developer feel in a programming language that has already decided on the builtin video codec. I think it has nothing to do in the core language, but in a library that show how powerful the core language feature is, yes of course. A programming language is not defined by the amount of featuers, but more by the amount of features it does not need because of more general core featuers that can be used to implement such a thing.

I am developing from time to time this project: https://github.com/krux02/opengl-sandbox

This has neither support for video loading, nor does it have support for audio. But I can imagine that at some point it will have exacty the feature you are proposing. But as I said, it should not be part of the standard library.

2017-01-22 20:02:47
If anyone's interested, I've started integrating webm support into the rod engine. The core libs are webm which is mkv/libvpx wrapper and rod_video which is a component for rod. All of the libs are still pretty immature, so PRs are welcome =)
2017-01-23 09:45:57
People in Audiovisual Computing need 2 things from a programming language:
  1. As fast as possible code execution speed
  2. Easy to use, built-in image/audio/video file read/write commands

There is NO programming language right now that does both.

The programming language is either SLOW but supports audio/video nicely, or it is FAST, but getting audio/video in and out of the language is a horror to set up.

We are in 2017. The world we live in is highly audiovisual content oriented.

If you want Nim to become popular in audiovisual computing - Computer Vision for self-driving cars or Video Processing for Film & Television for example - I strongly suggest that the language comes with basic image/audio/video read/write capabilities by default.

When people ask on the internet "what programming language is good for video processing?" people would then answer "Nim is really good for that".

Please consider basic audio/video support seriously.

Nim would become the first language that is both FAST, and that can do basic audio/video processing from the moment you install it.

You would reach thousands of users quickly just due to this built-in audiovisual functionality.

Thanks!

2017-01-23 13:36:28
Personally, i like the idea, and maybe i will try to help, but i'm not too familiar with video processing, so it takes time. 2017-01-23 15:21:10

As @krux02 mentioned, core language should be compact and flexible without diminishing its usability. Nim syntax already flexible enough so you shouldn't have any problem implementing the dsl for multimedia applications. While developing language to fill the niche is nice, it would be more nice if you can do anything with it.

Also, since the video/audio processing is your expertise, it would be nice if you write some library for it.

2017-01-23 15:21:31
ok, since I wrote this, I think the example with the codec developer is just a bad example. Just because there is a codec in the standard library, doesn't main you can't develop a better one in that language. But I don't change my opinion. I think the Nim language is powerfull enough, that it doesn't need any new language feature, but allows to implement all your dreams that you want to have. And sorry to tell you this, the Nim community is an idealistic underpaid small group of talented people. When you implement this idea in Nim, we will all support you in getting your things done. But we will not do it for you (unless you pay for the work you want to profit from). 2017-01-23 16:58:00

@videobuddha - what do you currently use (language/libraries/codecs/...)?

There are a few Nimble packages that wrap external libraries already:

audio
  • nim-ao
  • nim-ogg
  • nim-vorbis
  • nim-portaudio
  • nim-sndfile
  • sndhdr
  • nshout
  • taglib
  • sdl2 (for gaming)
video
  • nim-ogg
  • vidhdr
  • sdl2 (for gaming)

A generic video/audio module that abstracted lower level libraries would be of benefit (and is what I understand as being sort of what you are wanting).


If there was a programming language called XYZ that supported direct manipulation of video/audio files, then tomorrow, when I write my own unique fantasic (in my option) new shiny video/audio file format, then the XYZ language is obsolete and useless because it doesn't support by new shiny format (just trying to highlight that of course it is not the problem of XYZ that it doesn't support my new format).

So I agree with @Krux02

but (Nim) allows (you) to implement all your dreams that you want to have

It just needs support for more file formats, and a library to abstract the access to those libraries.

2017-01-24 00:02:58

@ jlp765

We video and image processing guys tend to be highly specialized in "pixel operations" and "image processing operations".

i.e. once we can get at the actual video RGB pixels or audio sample data we know what to do with them

But we suck at things like setting up video file I/O - getting a programming language to actually load and save video/audio data reliably

I have recently tried:

  • .NET languages with Mitov Videolab - quite easy to set up but SLOW processing speed
  • HLSL GPU shaders inside video players like Potplayer - unbelievably fast realtime processing, but currently no way to save video files

Many people who ask online "which language for video processing?" are typically told to look at Processing (slow) and Matlab (also slow)

Everybody is looking for the same thing - a programming language that is FAST and has easy-to-use audio/video data read/write functions.

If Nim could do this out-of-the-box, Nim would be 3 steps ahead of other programming languages when people look around for languages that can do audio/video.

This is just a suggestion of course - but it could be important if you want Nim to become a popular language for audiovisual computing tasks.

2017-01-24 16:04:32
<<<••12••>>>