Sketch2Code was launched by Microsoft AI Lab in 2018 as an open source project designed to help developers easily convert hand-drawn sketches into HTML code. Using Sketch2Code, developers can draw wireframe sketches and export them as source code.

Use artificial intelligence and automation to develop code from wireframe sketches, supporting basic web visual elements including labels, text fields, text paragraphs, images, buttons, and more.

When launched in 2018, the models behind the service had been trained on millions of images and could perform object detection on various types of objects. With this information, the tool can generate HTML code snippets for different elements in the design. It can then infer the layout of the design based on the positions of the identified elements and generate the final HTML code accordingly.

In the demonstration of GPT 4, there is also a demo demonstration of converting a sketch into a web page. It is likely that Microsoft has cooperated with OpenAI and iterated on this technology, and the latest training image data may also be far beyond 2018.