The generally accepted rule is that your bicycle must be sufficiently covered to prevent it from dirtying up your fellow passengers. Therefore, in the local vernacular, you have to "bag it".
Most good bike stores will sell nylon bike bags, or "rinko bukuro" ranging from Y3,000 to well beyond Y10,000. The majority fold down to a size that fits perfectly into a bottle cage which makes them convenient to carry when not in use.
Obviously your bike is big and the bag is small, so in general it is best to remove both wheels from the bike and strap them either side of your frame with the straps provided with your bike bag. You might like to place some cloth around your frame where it comes into contact with your wheels to prevent scratching or rubbing.
After your wheels are tightly secured, simply throw the bike bag over the whole package and run shoulder strap (also provided with your bike bag) from your headset to your rear fork or seat post for easier carrying.
If you find yourself without a bike bag and needing to ride the train in an emergency such as a sudden storm, strong winds or tropical downpour, then drop into a convenience store and buy some garbage bags. Bag each wheel and your frame in individually and you're good to go.
It has been rumored that there are cases in which you have to buy a special ticket to carry your bike on the train, but in 13 years of cycling around Japan I've never purchased one, so throw caution to the wind and only bother to buy one if you're asked by the station staff.
Also, on city trains try to ride in the front or rear most carriages as you'll find a little more space there for your bicycle. If you're riding the shinkansen then try to secure seats at the rear of any carriage as there is just enough space behind the rear seats to slide in a bagged bike.
If you're simply touring around the city I'd say avoid the trouble of bagging all together, as Tokyo is small enough that you can cycle from point A to point B in less time than it takes to bag your bike and take the train. Its only for those rides starting, or ending, outside the city that you should consider bagging your bike.
Update: After writing this article based on years of experience, rather than actual research, I came across Japan Railways (JR) rules for luggage which state there shall be no charge for:
Bicycles for use in cycling or sporting events if the bicycle is disassembled and placed in a bicycle carrying bag, or if the bicycle is a folding bicycle that has been folded and placed in a bicycle carrying bag.
The complete list of JR luggage rules can be found here.