Which has more information -- a one million bit file or a one bit file? If the million bits are completely random, they convey no information. If the one bit message is encoded as 0 signifies Republican and 1 signifies Democrat, we can note a person's political party with one bit.
Today's information technology coding schemes are all binary -- the codes are composed of 0s and 1s. Every storage and communication medium has its own way of representing 0s and 1s. Today they are invisible, but a few years ago we used media like punch cards, in which one could see whether or not there was a hole in a given position on the card:
There is a character in each column, so the capacity of a card was 80 characters. Each alphabetic character was represented by two punched holes, each numeric character 1 and the punctuation typically 3.
What is punched in this card?
My name punched in this piece of paper tape:
What is the code for the letter S?
In order to share information, we invent standard codes for different types of data. For example, we can use binary numbers for numeric data, ASCII codes for text, a color palette code for images and binary samples for audio data.
There are many ways to make 0s and 1s. In these examples, we used the presence or absence of a hole at a particular position on a card or paper tape. We no longer make bits with holes in cards and paper, we use other means -- a small spot being magnetized or not, a minute pit or smooth spot on a polished surface, a slight shift in the frequency, amplitude or phase of a radio wave, etc. As new, more efficient methods of recording and detecting 0s and 1s are invented, these will be replaced, just as holes in cards have been.
A final point is that we could have coding schemes that used more than two states, but today it is cheaper from an engineering standpoint to build two-state devices. That may change in the future as new technologies for recording and storing information are invented.