Let's teach a neural network to trace bitmaps

Flesh out your ideas for new or improved Inkscape features before submitting a request.
unfa
Posts: 6
Joined: Wed Mar 15, 2017 1:16 am

Let's teach a neural network to trace bitmaps

Postby unfa » Wed Mar 15, 2017 10:11 pm

If you have too much time on your hands and love developing cool stuff - this is probably for you :D

I've recently saw some examples of what deep neural networks can do. I wonder if they could be used to "intelligently" trace bitmaps creating vector shapes form low-res images?

Possibly preporcessing the image with a sobel & canny edge detector would be useful.

Probably such a neural network could "see" a fixed part of the input image (a circle of for example 8 pixels radius) and travel along the shape's edge, creating/refining bezier curve nodes and control points as it goes around.

Possibly useful resources:

https://www.youtube.com/watch?v=S9Y_I9vY8Qw - a neural network learning to play a racing game
https://www.youtube.com/watch?v=uihBwtPIBxM - finding edges in a bitmap: Sobel
https://www.youtube.com/watch?v=sRFM5IEqR2w - finding edges in a bitmap: Canny edge Detector
http://machinelearningmastery.com/tutor ... hon-keras/ - tutorial in creating a simple neural network in Python using Keras library

I guess training /evaluation data could consist of pairs of svg files of different black shapes on white background, and bitmap exports of these shapes.

The net would try to create a bezier path as close to the original one, and then a high resolution rendering of both (approximated and original beziers) can be pixel compared to define the score. Each different pixel increases the loss.

Example:

We draw 1000 random convex black shapes on white background in a 256 x 256 canvas.
All of these get rendered 1:1 and converted to grayscale PNG.
The neural net tries to approximate the beziers based on that low-res bitmaps.

Then we render the neural-net created beziers with 4x the resolution (1024x1024) and compare that to the 1024x1024 rendering of the original shapes.
Each pixel that isn't identical on both images increases the loss, so we can asses how well it did.

Tadam!

There are some holes to fill in here, but this at least can be fun. At most - it could become really useful one day.


User avatar
brynn
Posts: 10309
Joined: Wed Sep 26, 2007 4:34 pm
Location: western USA
Contact:

Re: Let's teach a neural network to trace bitmaps

Postby brynn » Thu Mar 16, 2017 8:58 am

For those of us who are not very technically inclined or aware - what is the benefit of tracing by a neural network and tracing by Trace Bitmap?

v1nce
Posts: 696
Joined: Wed Jan 13, 2010 4:36 am

Re: Let's teach a neural network to trace bitmaps

Postby v1nce » Fri Mar 17, 2017 4:37 am

with NN:
10x better
1000x slower
:D

tylerdurden
Posts: 2344
Joined: Sun Apr 14, 2013 12:04 pm
Location: Michigan, USA

Re: Let's teach a neural network to trace bitmaps

Postby tylerdurden » Fri Mar 17, 2017 5:23 am

:lol: :lol:

I gotta fast machine though, so it might be worth it.... Think how long cleaning up traces takes.

The code is young too, so maybe some streamlining will help.
Have a nice day.

I'm using Inkscape 0.92.2 (5c3e80d, 2017-08-06), 64 bit win8.1

The Inkscape manual has lots of helpful info! http://tavmjong.free.fr/INKSCAPE/MANUAL/html/

User avatar
brynn
Posts: 10309
Joined: Wed Sep 26, 2007 4:34 pm
Location: western USA
Contact:

Re: Let's teach a neural network to trace bitmaps

Postby brynn » Fri Mar 17, 2017 5:28 am

1000x slower? Would that be worth the effort? Or maybe we are talking about a cool toy, which the fun trumps the bumps?

unfa
Posts: 6
Joined: Wed Mar 15, 2017 1:16 am

Re: Let's teach a neural network to trace bitmaps

Postby unfa » Tue May 23, 2017 6:28 pm

Well, many of use have some GPUs that a framework like Keras / Tensor Flow should be able to use for reasonably fast NN deployment.

unfa
Posts: 6
Joined: Wed Mar 15, 2017 1:16 am

Re: Let's teach a neural network to trace bitmaps

Postby unfa » Tue May 23, 2017 6:52 pm

Actually right now tracing on my CPU usually takes a few seconds, if I had to sweat my GPU for a couple of minutes to get a 10x better result - I'm all for it!

alesg
Posts: 1
Joined: Tue May 30, 2017 6:51 am

Re: Let's teach a neural network to trace bitmaps

Postby alesg » Tue May 30, 2017 6:53 am

unfa wrote:Actually right now tracing on my CPU usually takes a few seconds, if I had to sweat my GPU for a couple of minutes to get a 10x better result - I'm all for it!



Hey Unfa, I am thinking about the same thing! Did you go further on the project?

best

unfa
Posts: 6
Joined: Wed Mar 15, 2017 1:16 am

Re: Let's teach a neural network to trace bitmaps

Postby unfa » Thu Jun 22, 2017 12:10 am

Well, I have no experience with neural networks, and have minimal experience with programming in general so I don't think I can develop anything as complicated.


Return to “Inkscape Ideas”