If you have too much time on your hands and love developing cool stuff - this is probably for you
I've recently saw some examples of what deep neural networks can do. I wonder if they could be used to "intelligently" trace bitmaps creating vector shapes form low-res images?
Possibly preporcessing the image with a sobel & canny edge detector would be useful.
Probably such a neural network could "see" a fixed part of the input image (a circle of for example 8 pixels radius) and travel along the shape's edge, creating/refining bezier curve nodes and control points as it goes around.
Possibly useful resources:
https://www.youtube.com/watch?v=S9Y_I9vY8Qw - a neural network learning to play a racing game
https://www.youtube.com/watch?v=uihBwtPIBxM - finding edges in a bitmap: Sobel
https://www.youtube.com/watch?v=sRFM5IEqR2w - finding edges in a bitmap: Canny edge Detector
http://machinelearningmastery.com/tutor ... hon-keras/ - tutorial in creating a simple neural network in Python using Keras library
I guess training /evaluation data could consist of pairs of svg files of different black shapes on white background, and bitmap exports of these shapes.
The net would try to create a bezier path as close to the original one, and then a high resolution rendering of both (approximated and original beziers) can be pixel compared to define the score. Each different pixel increases the loss.
Example:
We draw 1000 random convex black shapes on white background in a 256 x 256 canvas.
All of these get rendered 1:1 and converted to grayscale PNG.
The neural net tries to approximate the beziers based on that low-res bitmaps.
Then we render the neural-net created beziers with 4x the resolution (1024x1024) and compare that to the 1024x1024 rendering of the original shapes.
Each pixel that isn't identical on both images increases the loss, so we can asses how well it did.
Tadam!
There are some holes to fill in here, but this at least can be fun. At most - it could become really useful one day.
Let's teach a neural network to trace bitmaps
Re: Let's teach a neural network to trace bitmaps
tracing with nn
http://hi.cs.waseda.ac.jp/~esimo/public ... PH2016.pdf
tracing non NN
http://www.cs.northwestern.edu/~sco590/npar2011.pdf
http://chizhang.me/Publications/TVCG/TVGC_Vec.pdf
http://cg.cs.tsinghua.edu.cn/papers/TVC ... artoon.pdf
https://hal.inria.fr/hal-01309271/file/ ... licity.pdf
http://cg.cs.tsinghua.edu.cn/papers/TVC ... artoon.pdf
http://hhoppe.com/vectorization.pdf
https://pdfs.semanticscholar.org/b2d0/4 ... 5788c3.pdf
http://hi.cs.waseda.ac.jp/~esimo/public ... PH2016.pdf
tracing non NN
http://www.cs.northwestern.edu/~sco590/npar2011.pdf
http://chizhang.me/Publications/TVCG/TVGC_Vec.pdf
http://cg.cs.tsinghua.edu.cn/papers/TVC ... artoon.pdf
https://hal.inria.fr/hal-01309271/file/ ... licity.pdf
http://cg.cs.tsinghua.edu.cn/papers/TVC ... artoon.pdf
http://hhoppe.com/vectorization.pdf
https://pdfs.semanticscholar.org/b2d0/4 ... 5788c3.pdf
Re: Let's teach a neural network to trace bitmaps
For those of us who are not very technically inclined or aware - what is the benefit of tracing by a neural network and tracing by Trace Bitmap?
Basics - Help menu > Tutorials
Manual - Inkscape: Guide to a Vector Drawing Program
Inkscape Community - Inkscape FAQ - Gallery
Inkscape for Cutting Design
Manual - Inkscape: Guide to a Vector Drawing Program
Inkscape Community - Inkscape FAQ - Gallery
Inkscape for Cutting Design
Re: Let's teach a neural network to trace bitmaps
with NN:
10x better
1000x slower
10x better
1000x slower
-
- Posts: 2344
- Joined: Sun Apr 14, 2013 12:04 pm
- Location: Michigan, USA
Re: Let's teach a neural network to trace bitmaps
I gotta fast machine though, so it might be worth it.... Think how long cleaning up traces takes.
The code is young too, so maybe some streamlining will help.
Have a nice day.
I'm using Inkscape 0.92.2 (5c3e80d, 2017-08-06), 64 bit win8.1
The Inkscape manual has lots of helpful info! http://tavmjong.free.fr/INKSCAPE/MANUAL/html/
I'm using Inkscape 0.92.2 (5c3e80d, 2017-08-06), 64 bit win8.1
The Inkscape manual has lots of helpful info! http://tavmjong.free.fr/INKSCAPE/MANUAL/html/
Re: Let's teach a neural network to trace bitmaps
1000x slower? Would that be worth the effort? Or maybe we are talking about a cool toy, which the fun trumps the bumps?
Basics - Help menu > Tutorials
Manual - Inkscape: Guide to a Vector Drawing Program
Inkscape Community - Inkscape FAQ - Gallery
Inkscape for Cutting Design
Manual - Inkscape: Guide to a Vector Drawing Program
Inkscape Community - Inkscape FAQ - Gallery
Inkscape for Cutting Design
Re: Let's teach a neural network to trace bitmaps
Well, many of use have some GPUs that a framework like Keras / Tensor Flow should be able to use for reasonably fast NN deployment.
Re: Let's teach a neural network to trace bitmaps
Actually right now tracing on my CPU usually takes a few seconds, if I had to sweat my GPU for a couple of minutes to get a 10x better result - I'm all for it!
Re: Let's teach a neural network to trace bitmaps
unfa wrote:Actually right now tracing on my CPU usually takes a few seconds, if I had to sweat my GPU for a couple of minutes to get a 10x better result - I'm all for it!
Hey Unfa, I am thinking about the same thing! Did you go further on the project?
best
Re: Let's teach a neural network to trace bitmaps
Well, I have no experience with neural networks, and have minimal experience with programming in general so I don't think I can develop anything as complicated.