We all love LEGO, a truly generation-free entertainment that loads of people are obsessed about despite age.Although LEGO sets have entertained generations of children and adults, the challenge of designing customized builds matching the complexity of real-world or imagined scenes remains too great for the average enthusiast.
Researchers at the Massachusetts Institute of Technology (MIT) have developed an Image2Lego neural network algorithm that builds instructions for building a 3D LEGO model from a 2D image.
The process goes as following:
1. The user uploads a regular image: for example, with an airplane.
2. The algorithm recognizes the plane in the photo and uploads it to the Image2Lego neural network.
3. Trained Image2Lego transforms a 2D picture into a 3D model of an airplane using neural networks and shows how it should look if assembled from LEGO bricks.
4. The algorithm creates instructions for assembling the model and tells you what parts will be needed for this.
Here is how the researches described the objective: “We design a novel solution to this problem that uses an octree-structured autoencoder trained on 3D voxelized models to obtain a feasible latent representation for model reconstruction, and a separate network trained to predict this latent representation from 2D images.”
In the past we’ve already seen implementations of AI in LEGO assembling. For example, BrickIt allows a user to lay out all their LEGO pieces flat and scan with a mobile. The image then is processed by computer vision to detect available bricks suggesting possible models for assembling, both genuine and custom. However, the MIT’s neural network is quite opposite to what the BrickIt app offers and allows more usage scenarios.
Assembling a simple model like a tower or a car is just too banal. Image2Lego is capable of processing a face image to create detailed instructions for assembling a detailed 3D face. Trying it out for building a model of your own face must be a unique thing.
Here is Image2Lego’s instructions for assembling actor Chris Pratt’s face:
