moayuisuda

Recently, I worked on a small dapp project to consolidate my learning of smart contracts and also tried to use AI to enhance the customization and fun of NFTs. This article mainly records some key steps in the process.

Brain Storm#

I usually start with brainstorming, imagining the final effect, and then work backward to implement it.

1. Use AI to generate avatars. Simply generating avatars is a bit boring; the generated avatars can be related to the real-world state. I want a block to anchor the AI's black box output and the feeling of the real-world state.
2. Add a card-drawing element, where for AI painting, this "card" is defined as "painting keywords."
3. The generated avatars can be minted as NFTs. Simply minting is also a bit boring; it shouldn't just be an image on-chain; it can have more of a "genuine" feel. I am willing to spend money on genuine games and artworks, partly to support the authors and largely because of the subsequent supporting services. In addition to the inherent trading characteristics of NFTs, users after minting should have more additional extended functions.

Let’s briefly determine the user flow for the first version:

1. After entering, click the generate button (with optional configurations) to generate the avatar image.
2. After generation, you can choose one to mint as an NFT.
3. If you want to mint, click to connect your wallet and then click mint.
4. After minting is complete, minting information will be recorded as a basis for providing subsequent services (such as generating based on the minted image seed). At the same time, a certain number of points will be rewarded, and a painting keyword will be drawn.
   

Effect Preview#

GPU servers are very expensive. To prevent malicious DDoS attacks, the online trial is temporarily only open to a few friends. Here are some screenshots to see the process:

Initial state, the user has not connected a wallet, and only the "flower" keyword is available:

Click "make a wish" to generate an avatar.
Each avatar's result is somewhat related to the time and place of generation. For example, if you are in Japan and generate during the day, the character in the avatar will be dressed in a kimono during the day:

After obtaining the image, if you want to mint, you need to connect your wallet:

After successfully connecting the wallet and minting, you will receive a drawn painting keyword:

Returning to the homepage, you will find that you can generate avatar images based on the already minted images, and additional usable keywords have been added:

Let's try generating based on the previous avatar using the new keyword:

The new keyword "Mononoke Hime" comes from "Princess Mononoke," and the generated avatar will also have the style of Princess Mononoke:

The preview is complete; now let's talk about how each part of the application is implemented.

AI Avatar Generation Service#

The currently most popular open-source AI painting generation project is stable-diffusion-webui (hereinafter referred to as sd-webui). After deployment, it can directly start a local web UI service or run in API mode, exposing interfaces for other services to call. You just need to add the --api parameter when starting.
Clone the repository and install it, then run the command under the sd-webui repository:
./webui.sh --api
or
./webui.sh --nowebui
The difference is that --api provides API interface services while still starting the original local web UI service, whereas --nowebui only provides interface services.

After successful startup, open this webpage:
http://127.0.0.1:7860/docs, and you should see the corresponding API documentation:

Then you can call the sd-webui interface just like calling a regular interface. For example, the most commonly used text2img function:

The image resource returned by the interface is a base64 string, which is convenient for front-end processing. Additionally, you can utilize the randomness of the sd model's black box noise by hashing the base64 string or directly hashing the image seed, serving as a pseudo-random seed for other functions like lottery.
If cross-origin requests are needed after deployment, add the --cors-allow-origins parameter when running "webui.sh".

※ If there are too many issues during the installation process, you can try using this docker image directly: https://hub.docker.com/r/kestr3l/stable-diffusion-webui

Contract Writing#

The required functions for the contract:

1. Basic NFT functions (minting, transferring, associating tokenURI).
2. Record all NFT tokenIds owned by a user.
3. Points function (to be used in subsequent services).

This time, I chose to write the smart contract based on the ERC721Enumerable base contract. The contract code is as follows:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.7;

import "@openzeppelin/contracts/token/ERC721/extensions/ERC721Enumerable.sol";
import "@openzeppelin/contracts/access/Ownable.sol";
import "hardhat/console.sol";

error Insufficient_Mint_Value();

contract HimeNft is ERC721Enumerable, Ownable {
    uint256 private constant MINT_PRICE = 0.1 * 1e18;
    uint256 private _tokenId = 0;
    mapping(uint256 => string) private _tokenURIs;
    mapping(address => uint256) private _points;

    constructor() ERC721("HimeNft", "HN") {}

    function mintNft(
        string calldata tokenURI
    ) public payable returns (uint256) {
        if (msg.value < MINT_PRICE) revert Insufficient_Mint_Value();

        _safeMint(msg.sender, _tokenId);
        _setTokenURI(_tokenId, tokenURI);
        _tokenId += 1;
        _points[msg.sender] += 10;
        return _tokenId;
    }

    function withDraw() public onlyOwner {
        (bool callSuccess, ) = payable(msg.sender).call{
            value: address(this).balance
        }("");
        require(callSuccess, "call failed");
    }

    function _setTokenURI(
        uint256 tokenId,
        string memory _tokenURI
    ) internal virtual {
        require(
            _exists(tokenId),
            "ERC721URIStorage: URI set of nonexistent token"
        );
        _tokenURIs[tokenId] = _tokenURI;
    }

    function tokenURI(
        uint256 tokenId
    ) public view virtual override returns (string memory) {
        _requireMinted(tokenId);

        return _tokenURIs[tokenId];
    }

    function getTokenCounter() public view returns (uint256) {
        return _tokenId;
    }

    function getPoints(address owner) public view returns (uint256) {
        return _points[owner];
    }

    function getOwnedTokens(
        address owner
    ) public view returns (uint256[] memory) {
        uint256 tokenCount = balanceOf(owner);
        uint256[] memory result = new uint256[](tokenCount);
        for (uint256 i = 0; i < tokenCount; i++) {
            result[i] = tokenOfOwnerByIndex(owner, i);
        }
        return result;
    }
}

※ This is just a simple implementation. For future functionality upgrades, it is best to use a proxy contract to mediate interactions between the application and the contract.

Here is the test code for local testing:

const { assert, expect } = require("chai");
const { deployments, ethers, getNamedAccounts } = require("hardhat");

describe("HimeNft", async () => {
  const MINT_VALUE = ethers.utils.parseEther("0.1");
  const INSUFFICIENT_MINT_VALUE = MINT_VALUE.sub(1);
  const TEST_TOKENURI = "TEST_TOKENUR";
  let deployer, nft;

  beforeEach(async () => {
    await deployments.fixture();
    deployer = (await getNamedAccounts()).deployer;
    nft = await ethers.getContract("HimeNft", deployer);
  });

  it("initial token count is 0", async () => {
    const counter = await nft.getTokenCounter();

    assert.equal(counter, 0);
  });

  it("revert on mint value is insufficient", async () => {
    expect(
      nft.mintNft(TEST_TOKENURI, {
        value: INSUFFICIENT_MINT_VALUE,
      })
    ).to.be.revertedWith("Insufficient_Mint_Value");
  });

  it("add correct points after mint", async () => {
    await nft.mintNft(TEST_TOKENURI, {
      value: MINT_VALUE,
    });
    const points = (await nft.getPoints(deployer)).toString();

    assert.equal(points, "10");
  });

  it("get correct tokenURI and token list", async () => {
    const TEST_TOKENURI0 = "TEST_TOKENURI0";
    const TEST_TOKENURI1 = "TEST_TOKENURI1";

    await Promise.all([
      nft.mintNft(TEST_TOKENURI0, {
        value: MINT_VALUE,
      }),
      nft.mintNft(TEST_TOKENURI1, {
        value: MINT_VALUE,
      }),
    ]);

    const list = (await nft.getOwnedTokens(deployer)).toString();
    const tokenURI0 = await nft.tokenURI(0);

    assert.equal(tokenURI0, TEST_TOKENURI0);
    assert.equal(list, "0,1");
  });
});

But you may notice that there seems to be no logic for "drawing a painting keyword."
Remember the earlier mention of "directly hashing the image seed as a pseudo-random seed for other functions like lottery"? To save storage space, I store the image seed in the NFT's metadata and map the seed to a painting keyword. This way, minting can also complete the "drawing of a painting keyword" while saving gas.
For example, the simplest case, I have a map like this:

{
  "royal": 0,
  "cute": 1,
}

I process the seed using modulo. For example, if the seed is 334451, and we set the divisor to 10, taking the modulo mod(334451,10) gives a result of 1, which corresponds to the drawn result being the "cute" keyword. Of course, this mapping algorithm can be more complex, even configuring the probabilities of each item.
For sufficient transparency, this map method should be uploaded to the blockchain and verified.

The contract also defines a "_points" map to store points, which will be used as tokens for other services later (for simplicity, I did not use an additional ERC20 contract to implement tokens; if you want it to be a more "standard" token, you can implement it separately with ERC20 or change this contract to ERC1155 to manage multiple tokens).

Building the Frontend Interface#

This time, I used next.js + tailwind + react-moralis + ethers.js to build and deploy the frontend service.
I won't elaborate on this here; the code is available at https://github.com/moayuisuda/HimeAvatar.

It is worth noting that the useWeb3Contract hook from react-moralis supports batch calls:
You can directly call contract methods:

const { runContractFunction } = useWeb3Contract({
  abi,
  contractAddress: address[chainId as keyof typeof address],
  functionName: "mintNft",
  params: { tokenURI: "TOKENURL" },
});

runContractFunction()

You can also define the contract's ABI, address, and other basic information first, then call specific methods:

const { runContractFunction } = useWeb3Contract({
  abi,
  contractAddress: address[chainId as keyof typeof address],
});

runContractFunction({
  params: {
    functionName: "mintNft",
    params: { tokenURI: "TOKENURL" },
  },
})

With that, all parts of the setup are complete. You can package these services into a docker for easy one-click deployment on the server.

Todo#

• Use OpenAI's SDK to build a coherent worldview and dialogue (I personally recommend using the Python version of the SDK; I find the Node.js version a bit awkward), turning the simple avatar construction into character building.
• Support creators to upload seed images and generate Lora models, produce images, and share profits with creators.
• Build additional service stores for points tokens.

Gift#

The models used in the application are personally refined by me. The base model of the large model may have copyright issues, so generating avatar images is completely free.
Here are some nice avatars generated during previous refining; feel free to use them if you like: