• About
  • Home
  • Oracle Enterprise Manager

Musings

~ Things I see and learn!

Category Archives: Storage

Understanding Docker Images and Layers

08 Friday Sep 2017

Posted by adeeshfulay in Docker, Storage

≈ Leave a comment

Tags

cloud, Docker, Docker, oracle, stateful, weblogic

Not so long ago, I presented a couple of sessions at the IOUG Collaborate 17 conference. During my session ‘Docker 101 for Oracle DBAs’ there were a bunch of questions regarding Docker images and the concept of layers and their benefits. So this blog basically summarizes the discussion I had with the audience during my session.

What are Docker Images and how do they compare to VM Images?

docker layers1

For anyone who has used VMs, the concept of a VM image is not new. A Docker image is very similar and serves the same purpose, that is, they are used to create containers, but that is where the similarity ends. While VM image is a single large file, a Docker image references a list of read-only layers that represent differences in the filesystem. These layers are stacked one over the other, as shown in the image below, and form the basis of the container root filesystem. The Docker storage driver stacks and maintains the different layers. The storage driver also manages sharing of layers across images. This makes building, pulling, pushing, and copying of images fast and saves on storage.

How are Images used to create Containers?

When you spawn a container (docker run <image-name>), each gets its own thin writable container layer, and all changes are stored in this container layer, this means that multiple containers can share access to the same underlying image and yet have their own data state.

When a container is deleted, all data stored is lost. For databases and data-centric apps, which require persistent storage, Docker allows mounting host’s filesystem directly into the container. This ensures that the data is persisted even after the container is deleted, and the data can be shared across multiple containers. Docker also allows mounting data volumes from external storage arrays and storage services like AWS EBS via its Docker Volume Plug-ins.

How do I find the layers of an Image?

The older versions of Docker provided `docker images –tree` which would show the tree view of all images and layers, unfortunately in the absence of that we have to look for other options.

For illustration, I am going to use an Oracle WebLogic server image I built. This image was built using multiple other images, each with their own set of layers.

docker layers2
You can see the total size of the image, currently 1.62GB, and the image ID.

Next, we will use the `docker history <image>` command to see the layers of this image. This output below shows all the layers that make up the weblogic image, but what are all these commands in the 2nd last column? For this, we have to refer to the dockerfile. Every instruction in the dockerfile creates a new layer, and since we used multiple dockerfiles to create the different images, this view shows the aggregate of all instructions run to build the final image we are using. Here is a good reference on how to write good dockerfiles.

docker layers3

While this information is good, it still doesn’t give us the hierarchical or tree view of images and layers. For this, we will use a hack. Run the following command to download an image from DockerHub, that will print out the tree view we want.

>> docker run –rm -v /var/run/docker.sock:/var/run/docker.sock nate/dockviz images -t

docker layers4

This view shows the same set of layers as the history command, but it also shows the lineage. If I focus on just the hierarchy of images & layers used to build by Oracle Weblogic image, I see that I used 4 different dockerfiles to build 4 different images. In fact, the last two images were derived from the same set of shared layers up till layer ‘bd54831efb16 Virtual Size: 1.2 GB’, after which the tree splits. Below is the hierarchy of images I built and used.

docker layers5

Now that I know the hierarchy of images being used, I can run ‘docker history’ command for each image and see its individual layers and the dockerfile instructions used to create them.

docker layers6

Once I completed mapping all layers to their respective images, I end up with a breakdown as follows.

docker layers7

Success!!

If you want to learn more about Docker and other container formats, take a look at my blog – Containers Deep Dive – LXC vs Docker


Shameless Plug: 

I currently work for Robin Systems and we provide an excellent container-based platform for both stateless and persistent stateful applications, especially for Big data applications and relational and no-SQL databases. This platform includes the 4 key components required to run stateful applications – containers (Docker and LXC), a scale-out block storage, virtual networking, and orchestration, and the platform can be deployed both on-premises on commodity hardware, or on public clouds like AWS.

Try our free community edition!

 

Advertisement

Understanding Flash: Summary – NAND Flash Is A Royal Pain In The …

16 Saturday May 2015

Posted by adeeshfulay in Storage

≈ Leave a comment

flashdba

chaos-order

So this is it – the last article in my mini-series on understanding flash. This is the bit where I draw it all together in a neat conclusion that makes you think, “Yes! That was worth reading”. No pressure eh?

So let me start with the conclusion first: as a storage medium, NAND flash is a royal pain in the ass.

Chaos

Why? Well, let’s look back at what we’ve learned in the previous 9 articles:

  • Data stored in flash pages cannot be overwritten without an erase operation taking place first
  • Erase operations affect entire blocks of flash pages instead of just the page you want to erase
  • Erase operations are an order of magnitude slower than read and program (write) operations
  • Flash media wears out and is limited by the number of program and erase operations (P/E Cycles) possible
  • The performance of a flash package is unbalanced: for example,

View original post 953 more words

About Me

Product Manager at CA Technologies
Formerly, at Robin Systems, VMware, Oracle, mValent.

Interests: Startups, Containers, Patriots, Warriors, Python, Problem-solving, Biking,…

Views are my own.

Follow Me

Follow @AdeeshF

View Adeesh Fulay's profile on LinkedIn

Recent Posts

  • Create EM Jobs via EMCLI
  • Create Database using EMCLI Verbs
  • Understanding Docker Images and Layers
  • Linux: Analyze disk space full issues
  • Can Containers Ease Cassandra Management Challenges?

Categories

  • agent (1)
  • Data Clone and Refresh (1)
  • Database as a service (4)
  • Deployment Procedures (1)
  • Docker (2)
  • emcli (6)
  • host management (1)
  • job system (6)
  • Linux (1)
  • oracle enterprise manager (16)
  • private cloud (5)
  • Provisioning (1)
  • Self Update (1)
  • Storage (2)

Archives

  • October 2017
  • September 2017
  • March 2017
  • September 2016
  • May 2015
  • March 2015
  • February 2015
  • November 2014

EM12c Tweets

My Tweets

Blog at WordPress.com.

Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
  • Follow Following
    • Musings
    • Already have a WordPress.com account? Log in now.
    • Musings
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...