Older question I know, but certainly something that confused me as a newb!
Essentially, rank is a particular characteristic of each skill, representing the difficulty in learning it, either as a new one or developing one already partially trained. At first, new players will be working mostly with rank 1 skills, and only much later the high rank skills required for capital ships, for example.
As explained above, this is represented in game by the training multiplier, which doubles, triples, quadruples (up to x16) the length of time which is will take to train the skill. It does this by multiplying the number of skill points it is necessary to accrue before the skill is rewarded. If a rank 1 takes 10 minutes to train to its first level, for example, a rank 16 skill based upon the same character attributes will take 160 minutes to do the same, for example.
Level is a personal measure, representing the degree to which a character is trained in any particular skill, on a scale between I, essentially beginner, and V, expert (seperately, rank is generally notated in Arabic numerals, level in Roman, which helps to distinguish them).
As mentioned above, the amount of training between each level of a skill becomes progressively harder, again by escalating the skill point requirement, so that the rank 1 skill you trained to level I in ten minutes, will then take you an hour to train from level I to II, and finally the best part of a week to train from level IV to V. If you're training a rank 16 skill to level V, having space left in your queue is the last thing you need worry about, it's going to take months, just for that one progession!
Level serves two main in game functions. It is often a prerequisite for being able to use a particular item, or train a follow on skill. Simply having Refining trained to level I is not enough, for example. It needs to be at level IV expertise before you can then train Scordite Processing, and level V before you can train Refinery Efficiency. Similarly, certain ships, for example, can not be flown until you have pushed your faction's Cruiser skill up to level III.
The other benefit of level is the modifiers they give you. Each level of Refining cuts 2% off the waste lost at the refinery. By training up to level V, you reduce your refinery waste by 10%. Ships are often focussed towards certain activities and skill sets by granting certain benefits per level of a certain skill.
It's up to you to decide whether the benefit of moving up a level outweighs the much longer time spent training it, especially in high rank skills, and whether a similar effect can be achieved by other means - training all of the ore processing skills to level I, for example, takes much less time than just training Refining that last stage from IV to V, and yet grants a 5% waste saving for each ore, rather than the just general 2% of the latter.