Digital image processing:  p006 - Image formation - Sampling  Quantization

Digital image processing: p006 – Image formation – Sampling Quantization

we're going to now talk about how an image is representing the computer how a color image is representing the computer and how a video is representing the computer and the very important concepts of sampling and representation and quantization it's always very good to have in mind the images that we normally take with our cameras but please remember what we see here and we talked a few videos ago that images that we are familiar with it's just a very small part of the spectrum of images that we can acquire at different wavelengths and with different modalities and the concept of something and quantization are relevant for basically all of those modalities we always going to have to sample we're always going to have to quantize but again is good to have in mind when we talk about these different ideas how they apply to the images that we are most familiar with which are the images from our own digital cameras so the basic idea is as follows we then say have light basically reflecting on the objects and the light that is reflected on the objects goes into the camera and what we have are sensors in the camera as we can see here remember in our eyes we have the retina and the retina are the cones and rods those were the sensors in digital cameras we have sensors exactly the same way as we have sensors in our eyes so this is just one example of a sensor the energy is coming in there are filters and then basically the energy is acquired and is transmitted so the energy sends and transmitted and normally what we are going to have on a regular camera is a two-dimensional array of these sensors so each one of these is going to be one of these okay and normally they are arranged in a square or rectangular fashion as we see it here so we have actually kind of a discreet number and that's when we go and buy digital cameras we hear that this is about 1,000 by 1,000 it means actually we're going to talk a bit more about that in a second it means that we have 1,000 sensors in this direction 1,000 sensors this direction so a total of 4,000 squared sensors so once again light is reflecting objects is projected into the camera and is sensed by the camera but all the process has a step of this protis ation some of you might be more familiar than others with these those of you that took classes in in digital signal processing and are aware of A to D converters this will be very simple and very familiar to you but let's just review it for those that are not so familiar with the concepts so we see here the light represented here is reflected from the object and it goes into the sensing device think about your digital cameras so this is a blown-up of what we have back here it looks the object is continuous base everything is continuous but we actually get a discrete representation of that object and the discretization applies in two different directions one is a spatial discretization so we have a discrete number of sensors here and basically the light one of the simplest modes is that we integrate each one of these integrates the light arriving to it so that's a discretization in the space there's also discretization in the amplitude we're not going to be able to capture every single level of gray every single level that is reflected from the object but we're gonna have to discretize it as well so let's see let's talk a bit more about that this is the object as we actually see it until it just look at one line across this object so we see that there it has different values it goes from white to gray so let us now draw the profile which is what we see here so it's very light white and then it becomes a bit dark and if we see here let me just erase this so we can basically see it more clearly if we go around I'm going to just mark it here if we're going to route we see that is going for white then it's dark then it just becomes brighter a bit and goes down again and then it's white again so it has kind of this profile hi is white then it got dark and then goes up down and it becomes white again of course it's not perfect it has a noise even when we go on by paint and paint a wall basically we think that is exactly the same color that's actually not it has some small variations which are represented here so again there are two processes happening here one is sampling we are going to basically look at the values at certain discrete intervals we are not going to have the values all around we'll have it at certain intervals okay that's one discretization the second discretization we're not going to be able to represent every single possible value here we're going to basically just represent a few of them we are going to round them so let's just give an example a very simple illustration how can we round let's say that we see the value 17.5 so basically we see that this is the value that is representing a particular gray value here so how can we round this one way is just to truncate it like this and then basically say you know I know that what came into the sensor is 17.5 by children are represented by 17 okay so there are more ways of doing this and we're going to talk in detail about quantization this is sampling and disposition in this direction of the great values is called quantization we could also consider for example we had 17.5 as before we could actually divide it by two take this and multiply by two so basically we divided by 2 then we round and then we multiply by 2 and I'm going to just ask you as a very simple exercise what's this value and what will happen if instead of dividing by 2 we basically for example divide by 4 run down and then multiply by four and I'm gonna have you think about this is a simple arithmetic operation we're going to discuss a lot about this particular operation in the chapter on quantization but to recap we have two ways of sampling one is the two basic ways of this creation one is something in the special domain and the other is quantization in the gray values so the basic idea is that while we have started from a continuous object the basically lives in the whole space and has multiple gray values wind up with a discrete object that has discretized discretization both in the spatial domain and also in the gray values the holiday of good digital cameras is to have a dense enough sampling here and very very fine level of quantization so we don't notice that everything is discrete but it's actually it is the screen so we have pixel values each one of these it's called a pixel this comes from picture element and is denote as pixel and actually I'm going to just if you want a Google or you want to search on the web a bit I'm going to just you might find it entertaining to find out who please it was the first person to use the term pixel is actually very famous statistician and you can have fun looking for that on the internet or discuss about that on the in the blogs when you find out about that so every one of these is the screen both in space and is discrete also in its grave Alex and that's how we get pixels and once again the more we have of this and the more we have of the levels the better the image it is so regular images will be for example 256 by 256 spatial domain that will means that you will have 256 square pixels and maybe it's represented by 8 bits so every pixel is represented by 8 bits which means that it can't have it can represent 256 different gray values and we are going to see how important this is both the spatial discretization and the quantization of gray values are extremely important we can have better images we're for it let me just go back we are going to have for example we might have an image that has instead of 256 by 256 you might have 512 by 512 and still 256 levels so it's going to take more memory to save that image but it's going to have a better quality and once again when you're buying digital cameras they normally tell you a lot about this number how many pixels it has and that tells you how fine the sampling is we want a lot of pixels so they are so close to each other that we that we can actually believe that is a continuous domain and the same at the level of the gray values so once again when each one of this is a pixel and is represented by gray values now this is a black and white image we actually in our cameras we have color images color images are represented by three of these images one for red one for green and one for blue so basically the cameras acquired I'm going to explain a bit more about that in a second acquire three images one represents the blue colors how much blue there is in the scene that represents the green and the other represents the red and this is called red green and blue so you often going to hear images which are our G and B I should make one comment about this most low end to meet and consumer cameras the actually don't really capture full two-dimensional raise for each one of the colors so we would like to have each one red green and blue plus what we would like to have a red green and blue tie would be the ideal scenario and the more expensive cameras actually do that basically the capture three full images most consumer cameras if you buy a low end digital camera actually does something slightly different something that is called mosaic and the basic idea is that it captured the pixels with interleaving in the colors will capture a red a green a red a green and so on and here it will capture a red here a blue blue red and so on so the next one will be red green red green the next one will be blue red blue red so actually it's not having did triple the amount of pixels is actually having one picture red one green one red one green from these we're going to have to interpolate and that we're going to learn how to do those things in the third and fourth week weeks in this class we're going to have to go because to represent a full color image at every picture we're going to have to go red green and blue so we do have the red we're going to have to create somehow the green and blue and that's done from the neighboring pixels here we have the green we're going to have to create the red and the blue so we are going to have to complete the three channels to be able to see a color image when you buy digital cameras they actually normally tell you the size on this which is the size after they have done that interpolation although they as I say the regular consumer images capture these so they actually capture one-third of the data that they tell you that they're giving to you that the data they tell you they are given to you has been interpolated from what has been captured so we have this images we have a red green and blue channels we're going to think about them as red green and blue so we already went from this representation to the interpolated by the way this is called mosaic so that's how color images are created now how are videos so this is how you have a color in your an image in your computer you have three channels three arrays of two dimensional discrete values one for red one for green one for blue so what happened with videos videos basically repeat this so one image has red green and blue and what happens in video you have multiple of these for example for one second of video in what in some of the format's you're going to have 30 of these so you're going to have 30 times R G and blue and each one of these remember is one of these two dimensional structures so basically if you are having if you want to have one second of video in your computer you have 30 times three images 30 images in one second and three because you have three colors so in one second you're going to have 90 images of this type to represent one second of video and of course 90 images so let's say 90 images at a resolution of 512 times 512 each and 8 bits each so this is what you need to represent at the relatively low-resolution this is not very high resolution this is pretty standard but basically this is what you're going to need to represent one second of a video in your computer by the way instead of saying that is three times a beats three times because we have our to be sometimes people will say that our resolution is 24 bits so we use 24 bits to represent basically a pixel okay so this is how you have if you get gone out to a computer and knows how to deal a program that knows how to deal with images you're going to see basically two dimensional discrete arrays as many as seconds or frames you have in your image each one of these 30 is called a frame and you have 30 of these in a second or 30 images when we talk about still images with top normal about images when we talk about videos each one of these we call it a frame so this is how your images are represented let's just look a bit more about how important and what are the effects of your resolution in space and your resolution in gray values meaning how many pixels you're going to have and what are the gray values that you're going to use for them so this is again the same type of representation this is how one color of one image or one color of one frame is represented if you have the letter D then these pixels are 0 when we have 8 bits normally 0 is black and 1 or 256 255 actually is white so if you are going with a beats you can go from 0 to 255 0 is black 255 is white sometimes it 255 is called 1 if you want to talk about the interval between 0 and 1 so although you see this actually in the computer is represented by a bunch of zeros for the black region then let's say that this region is in the middle of the gray value it will be an equivalent of a 128 value or 0.5 value so we see here 0.5 and then 1/4 basically the whites so these is your discrete representation of this image in the computer okay at every pixel you have a discrete value now sometimes one of the things that happen is because you have only 256 different levels that you can use sometimes your image is too bright and you might not be able to represent the small variations in brightness there remember the human visual system can basically adapt to multiple level to multiple levels of blindness but not all of them at the same time the same for an image we can capture many many levels maybe 256 but if we have to deal with a very dark region and a very light region at the same time we are in trouble and sometimes the very small variations in this light region we are not going to be able to capture and that's called saturation so it's going to look all white because it all got to the 255 maybe it was above 255 he wanted to be above 50 to 55 because it was very bright so that happens very often we take images and standards we see what's called specularity or or very strong reflections and it looks white it looks very very light and that's because the image the sensor has saturated so this explains now you know why is why that is happening you have a discrete amount of levels a discrete amount of values to represent your image and this one above and beyond what you can represent with that discrete amount of values so again to these predecessors one in space and one in gray values and look what happened what's the effect and you know that because if you have a digital camera or if you see images on the on the web on the Internet you see which is a different resolution so if you were to use a lot of pixels you might see this so this means that you have something which is very very fine a lot of pixels and that looks really nice on the other hand if you have something which is much more coarse less dense so you have less pixels you're going to see something like this and it's very clear to see for example how you know this line is very nicely represented here is not so nicely represented here you can see very easily the small numbers here not so we see here look at these numbers here that not so clear here so here we only have changed this spatial resolution we have not changed the number of gray values still let's say we have 256 so each one of these is represented with 256 and each one of this is representative 256 but we are not able basically to represent so fine detail with this as we are with this and you can think about once again if I want to draw a profile if you want to represent something that has this profile of gray value and you actually want to discretize that if you do it very course you're not going to be able to detect that there is a smooth variation in the gray values but if on the other hand you're able to put many more layers you're going to be able to capture its value at many many points instead of just add a few let me just use a different color just as a few values if you do a course discretization and that's why this image doesn't look as nice as this one because this capture was was going on at a very coarse levels in the discretization in the space discretization now similar things happen if you basically do a very coarse discretization of the gray values so let's start and once again we are not changing the number of pixels all these four images and the four images that are going to come next have exactly the same number of pixels okay so actually it's written here it has four hundred and fifty-two I'm going to just make it larger times 370 for all the images but the first image is displayed with 8 bits meaning 256 values so I'm going to just write it maybe I'm going to just save here the first is 256 and we actually see it very very smooth and very very nice here we only use 7 bits or 128 values so basically the value c1 1 becomes 0 2 & 3 become 2 4 & 5 become 4 okay the easiest way to do that is you take every value here let's say a value you divide by two you take basically the closest integer from below and you multiply it again by two okay so for example if the value of this image was eight you divide by two you take / to basically divide sorry / 2.4 okay you multiply by 2 you got 8 again no problem with that pixel value okay but what happened let's just do that here what happen if you have 9 you divide by 2 that's four and a half I have to take the closest integer from below that's four I multiply by two and it became eight so a nine was transformed into an eight and that basically reduces the range and reduces the quality of the image this becomes even worse if we go into this region where we are actually only using 64 different values so that's equivalent basically to dividing by 4 then truncating and multiplying by 4 and that means that you're going to start talking about basically if you have only 64 every 4 values are represented with one number so 0 1 2 3 they all become 0 4 5 6 7 they all become 4 and so on and then you can keep going and this is only represented with 32 so I hope that you're starting to see that the quality is deteriorating in particularly if you look at lines here you start to see what's called false contours you see these lines that they're not clear here there is a smooth transition here but here you see this false contours and that's created because of these jumps that I mentioned to you so if you were going basically 1 2 3 4 it's actually you're gonna when you start taking very very few gray values you're going to go 0 0 0 0 4 you're going to jump into that value and that's because you're basically have much less resources to represent the different gray values let's just go in into an even more extreme case and we keep going down here basically we use only 16 and now I think that you're going to be able to see the false contours very clearly here we go basically only to eight values there's only 8 numbers to represent everything that is happening in this picture ok and that's basically a very strong quantization here there is basically only four and here is basically only two in this very extreme case everything that was below 128 becomes zero we can black everything that was above 128 became 255 why and then you see a very very poor quality data okay so that's how the spatial quantization and the gray value is spatial sampling sorry and the gray value quantization affect you the more you have of samples the better quality it is the more you have the possibility of representing many many many small variations in gray values the better it is of course the more memory it takes into your computer and that's going to be the subject of our compression unit so now we know how images are representing the computer that represent us two-dimensional arrays gray level images are represented by one color images are represented by three red green and blue videos are represented by multiples of them let's say 30 a second sometimes is only 24 depending on the standard that that we have but there are multiple of those images and as we say we call them frames and we know that everything is discrete so we have sampling and quantization an image in a computer is nothing else now that the two-dimensional array of discrete values thank you

16 thoughts on “Digital image processing: p006 – Image formation – Sampling Quantization”

  1. Great lectures, thanks !_!

    For the RGB interpolation of images, this doesn't necessarily have to be done as they could potentially be stored in mosaic form at 1/3 the size. Is this done for compatibility's sake, or to make processing the images faster (such as when a video/image is being displayed)? Or is there some other reason why the interpolated RGB form is used over the mosaic form, at the cost of the extra file size.

  2. first of all thanks for the class. 
    I have a question, around 19 min you say that images also adapt depending on the  brightness level, does that mean that if the flower image didnt had a black background it would have a better definition of the white levels? does the 256 values depends on the lower and highest level of light of that specific picture or is 0 always black and 255 the whitest you can get?
    thanks again

Leave a Reply

Your email address will not be published. Required fields are marked *