Product is the overall name used for items on the site.
Product Display is a combination of the displays of the product variations via term reference and the product content type.
Category is the taxonomy term or terms used to tag the products ie. to wear, to drink, to carry, etc.
Line item type refers to the a line item of product, shipping, discount or other needed line item type.
Variation type is the different variations of the product like colors or sizes.
As for best practices the tutorial her on Drupal Commerce are very good. https://drupalcommerce.org/commerce-kickstart-2
As for product creation workflow this is a very wide subject, you can import products, bulk create them or make them one by one. Personally I prefer to import them and use the csv to upload on Google shopping and other shopping sites that will accept uploads or feeds.
Hope this helps you and gets you started in the right direction.. Good luck