text:
Oracle Linux 5.9
By Wcoekaer-Oracle on Jan 16, 2013
ISO images will be available shortly from http://edelivery.oracle.com. If there is an urgent need to get the ISOs through My Oracle Support, simply file a service request.
Release notes are here.
ISO images will be available shortly from http://edelivery.oracle.com. If there is an urgent need to get the ISOs through My Oracle Support, simply file a service request.
Release notes are here.
To write your own module, you have to build an RPM that contains a configure script in a specific format, let's go through the steps to do this.
Oracle VM template configure works very similar to the init.d and chkconfig script model. For template config we have the /etc/template.d directory, all the scripts go into /etc/template.d/scripts. Then symlinks are made to other subdirectories based on the type of target the scripts provide. At this point we handle configure and cleanup. When a script/module gets added using ovm-chkconfig, the header of the script is read to verify the name, priority and targets and then a symlink is made to the corresponding subdirectories under /etc/template.d.
As an example, you have /etc/init.d/sshd which is the main sshd initscript and when sshd is enabled you will find a symlink in /etc/rc3.d/S55sshd to /etc/init.d/sshd. These symlinks are created by chkconfig when you enable or disable a service. The same thing goes for Oracle VM template config and the content of /etc/template.d/scripts. You will see /etc/template.d/scripts/ssh and since ssh (on my system) is enabled for the configure target, I have a symlink to /etc/template.d/configure.d/70ssh.
Like init.d, the digit in front of the script name specifies the priority at which it should be run.
The most important and complex part is writing your own script for your own application. Our scripts are in python, theoretically you could write it in a different language, as long as the input, output and argument handling remains the same. The examples here will all be in python. Each script has 2 main part : (1) the script header which contains information like script name, targets, priorities and description and (2) the actual script which has to handle a small set of parameters. You can take a look at the existing scripts for examples.
(1) script header
Aside from a copyright header that suits your needs, the script headers require a very specific comment block, here is an example :
### BEGIN PLUGIN INFO # name: network # configure: 50 # cleanup: 50 # description: Script to configure template network. ### END PLUGIN INFO
You have to use the exact same format. Provide your own script name, which will be used when calling ovm-chkconfig, the targets (right now we implement configure and cleanup) and the priority for your script. The priority will specify in what order the scripts get executed. You do not have to implement all targets, if you have a configure target but not cleanup, that is OK, same goes for cleanup versus configure. It is up to you. The configure target gets called when a first boot/initial start of the VM happens, cleanup happens when you manually initiate a cleanup in your VM or when you want to restore the VM to its original state.
### BEGIN PLUGIN INFO # name: [script name] # [target]: [priority] # [target]: [priority] # description: a description and can # cross multiple lines. ### END PLUGIN INFO
Now for the body of the script. Basically the main requirement is that it accepts a [target] parameter. Let's say we have script called foo that needs to be run at configure time, then the script (/etc/template.d/scripts) will have to accept and understand handling the parameter configure. If you also want to call it for cleanup, then it has to handle cleanup. You can have your script handle any other arguments, this is totally up to you, they are optional for our purposes. There is one optional parameter which is useful to implement and this is -e or --enumerate. ovm-template-config uses this to be able to enumerate the parameters for a target for your script.
Here is the firewall example:
# ovm-template-config --human-readable --enumerate configure --script firewall [('41', 'firewall', [{u'description': u'Whether to enable network firewall: True or False.', u'hidden': True, u'key': u'com.oracle.linux.network.firewall'}])]and if you run the script manually :
# ./firewall configure -e [{"hidden": true, "description": "Whether to enable network firewall: True or False.", "key": "com.oracle.linux.network.firewall"}]
In other words, the firewall script lists the parameters it expects when run as a configure target.
Now here is an example of the script body, in python. It implements the configure and cleanup target and handles the enumerate argument. Part of the magic is handled in templateconfig.cli.
try: import json except ImportError: import simplejson as json from templateconfig.cli import main def do_enumerate(target): param = [] if target == 'configure': param += [] elif target == 'cleanup': param += [] return json.dumps(param) def do_configure(param): param = json.loads(param) return json.dumps(param) def do_unconfigure(param): param = json.loads(param) return json.dumps(param) def do_cleanup(param): param = json.loads(param) return json.dumps(param) if __name__ == '__main__': main(do_enumerate, {'configure': do_configure, 'cleanup': do_cleanup})
So now you can fill this out with your own parameters and code. Again taking the firewall script as an example, to add expected keys :
def do_enumerate(target): param = [] if target == 'configure': param += [{'key': 'com.oracle.linux.network.firewall', 'description': 'Whether to enable network firewall: True or False.', 'hidden': True}] return json.dumps(param)
The above shows that this script expect the key com.oracle.linux.firewall to be set and what the default is, along with a description. Add this for each key/value pair that you expect for your script and then afterwards it is easy to understand what the input to your script needs to be, again by running ovm-template-config.
To execute actions at configure time, based on values set, here's a do_configure() example:
def do_configure(param): param = json.loads(param) firewall = param.get('com.oracle.linux.network.firewall') if firewall == 'True': shell_cmd('service iptables start') shell_cmd('service ip6tables start') shell_cmd('chkconfig --level 2345 iptables on') shell_cmd('chkconfig --level 2345 ip6tables on') elif firewall == 'False': shell_cmd('service iptables stop') shell_cmd('service ip6tables stop') shell_cmd('chkconfig --level 2345 iptables off') shell_cmd('chkconfig --level 2345 ip6tables off') return json.dumps(param)
When the script is called, you can use param.get() to retrieve key/value variables and then just make use of it. Just like in the firewall example, you can do whatever you want, call out other commands, add more python code, it's up to you...
It is also possible to alter keys or add new keys which then get sent back. So if you want your script to communicate values back which can be retrieved later through the manager API, for instance with ovm_vmmessage -q, you can simply to this :
param['key'] = 'some value'
Key can be an existing key, or a new one.
And that's really it... for the script. Next up is packaging.
In order to install and configure these template configure scripts, they have to be packaged in an RPM, with a specific naming convention. Package the script(s), there can be more than one, as ovm-template-config-[scriptname]. Ideally in the post install of the RPM you want to add the script automatically. Execute # /usr/sbin/ovm-chkconfig --add [scriptname]. When de-installing a script/RPM, remove it at un-install time, # /usr/sbin/ovm-chkconfig --del [scriptname].
Here is an example of an RPM spec file that can be used:
Name: ovm-template-config-example Version: 3.0 Release: 1%{?dist} Summary: Oracle VM template example configuration script. Group: Applications/System License: GPL URL: http://www.oracle.com/virtualization Source0: %{name}-%{version}.tar.gz BuildRoot: %(mktemp -ud %{_tmppath}/%{name}-%{version}-%{release}-XXXXXX) BuildArch: noarch Requires: ovm-template-config %description Oracle VM template example configuration script. %prep %setup -q %install rm -rf $RPM_BUILD_ROOT make install DESTDIR=$RPM_BUILD_ROOT %clean rm -rf $RPM_BUILD_ROOT %post if [ $1 = 1 ]; then /usr/sbin/ovm-chkconfig --add example fi %preun if [ $1 = 0 ]; then /usr/sbin/ovm-chkconfig --del example fi %files %defattr(-,root,root,-) %{_sysconfdir}/template.d/scripts/example %changelog * Tue Mar 22 2011 Zhigang Wang
- 3.0-1 - Initial build.
Modify the content to your liking, change the name example to your script name, and add whatever else dependencies you might have or whatever files need to be bundled along with this. If you want to bundle executables or scripts that live in other locations, that's allowed. As you can see from the spec file, it automatically called ovm-chkconfig --add and --del at post-install and pre-uninstall time of the RPM.
In order to create RPMs, you have to install rpmbuild, # yum install rpm-build.
To make it easy, here's a Makefile you can use and help automate all of this :
DESTDIR= PACKAGE=ovm-template-config-example VERSION=3.0 help: @echo 'Commonly used make targets:' @echo ' install - install program' @echo ' dist - create a source tarball' @echo ' rpm - build RPM packages' @echo ' clean - remove files created by other targets' dist: clean mkdir $(PACKAGE)-$(VERSION) tar -cSp --to-stdout --exclude .svn --exclude .hg --exclude .hgignore \ --exclude $(PACKAGE)-$(VERSION) * | tar -x -C $(PACKAGE)-$(VERSION) tar -czSpf $(PACKAGE)-$(VERSION).tar.gz $(PACKAGE)-$(VERSION) rm -rf $(PACKAGE)-$(VERSION) install: install -D example $(DESTDIR)/etc/template.d/scripts/example rpm: dist rpmbuild -ta $(PACKAGE)-$(VERSION).tar.gz clean: rm -fr $(PACKAGE)-$(VERSION) find . -name '*.py[cdo]' -exec rm -f '{}' ';' rm -f *.tar.gz .PHONY: dist install rpm clean
Create a directory, copy over your script, the spec file and this Makefile. Run # make dist, to create a src tarball of your code and then # make rpm. This will generate an RPM in the RPMS/noarch directory. For instance: /root/rpmbuild/RPMS/noarch/ovm-template-config-test-3.0-1.el6.noarch.rpm
Next you can take this RPM and install it on a target system.
# rpm -ivh /root/rpmbuild/RPMS/noarch/ovm-template-config-test-3.0-1.el6.noarch.rpm Preparing... ########################################### [100%] 1:ovm-template-config-tes########################################### [100%]
And as you can see, it's added to the ovm-chkconfig list :
# ovm-chkconfig --list|grep testtest on:75 off off on:25 off off off off
One point of caution : the configure scripts get executed very early on in the bootstage. ovmd is executed as S00ovmd. This is well before many other services are (1) configured, (2) running. So if your product requires services like network connectivity or others to be up and running, then you have to split up the configuration into two parts. First, use the above to gather configuration data remotely, store it in a way that you can use it, and then add your own /etc/init.d scripts which can take this data afterwards. So you can have your own init scripts executed at a late stage when the services you depend on are available.
That's really all there is to it. Thanks to Zhigang for example code I have used here.
There are two ways to communicate through the APIs, a quick and easy one and a more comprehensive one.
The quick and easy method of just sending and receiving messages.
# ssh admin@localhost -p 10000 admin@localhost's password: OVM> sendVmMessage Vm name=ol6u3apitest key=foo message=bar log=no Command: sendVmMessage Vm name=ol6u3apitest key=foo message=bar log=no Status: Success Time: 2012-12-27 09:04:29,890 PST
# ./ovm_vmmessage -u admin -p ######## -h localhost -v ol6u3apitest -k foo -V bar Oracle VM VM Message utility 0.5.2. Connected. VM : 'ol6u3apitest' has status : Running. Sending message. Message sent successfully.
so both of the examples send a key/value pair of foo=bar to the VM.
Inside the VM, you can use the ovmd executable to retrieve and send messages back.
# ovmd -l
lists all currently set key/value pairs
# ovmd -g keyget a value from inside the VM
# ovmd -r keydelete a key out of the current cache
# ovmd -xdelete the key/value values currently set in the cache
so in the case of the message sent through the manager above, you can see it in the VM :
# ovmd -l {"foo":"bar"} {"mykey":"myvalue"} # ovmd -g mykey myvalue # ovmd -r mykey # ovmd -l {"foo":"bar"}
# ovmd -p key=valueset a key/value pair inside the VM
# ovmd -p newkey=newvalue # ovmd -l {"foo":"bar"} {"newkey":"newvalue"}
use ovm_vmmessage to query the value of a key, use the -q option with the key to query to retrieve a set value.
ovm_vmmessage will also return when this key was set inside the VM.
# ./ovm_vmmessage -u admin -p Manager1 -h localhost -v ol6u3apitest -q newkey Oracle VM VM Message utility 0.5.2. Connected. VM : 'ol6u3apitest' has status : Running. Querying for key 'newkey'. Query successful. Query for Key : 'newkey' returned value 'newvalue'. Key set 2 minutes ago.
So with just these simple tools it's possible to set up a model where you send message from your application outside of a VM to the VM through the OVMAPI and also send messages from an application inside a VM back. You can write your own daemon process that runs and queries for values or just do it manually. A recommendation would be to create a naming convention for your product. For instance, for the Oracle VM template configuration we use com.oracle.linux.[values]. You could consider something similar or just have [application].[key] or whatever you want. The maximum size of the total message is 8Kb.
Next up will be extending template config for your own application.
By introducing the offline client, customers with Oracle Linux Premier or Oracle Linux Premier Limited support can create a local intranet yum repository that creates a local mirror of the ULN ksplice channel and just use the yum update command to install the latest ksplice updates. This will allow customers to have just one server connected to the oracle server and every other system just have a local connection.
Here is an example on how to get this going, and then setting up a local yum repository is an exercise up to the reader :)
Now, as we release new zero downtime updates, there will be a newer version of the uptrack-updates RPM for your kernel. A simple # yum update will pick this up, (without the need for a reboot) and the update will apply new ksplice updates. You can see the version of the uptrack-updates by doing # rpm -qa | grep uptrack-updates it will show the date appended at the end. And of course the usual commands # uptrack-show or # uptrack-uname -r.
The offline client allows you to create a local yum repository for your servers that are covered under Oracle Linux Premier and Oracle Linux Premier Limited support subscriptions so that the servers just point to your local yum repository and all you have to do is keep this repository up-to-date for the ksplice channel and install the above RPMs on each server and run a yum update. In other words, the offline client doesn't require each server to be connected to the Oracle Unbreakable Linux network.
To set up a local yum repository, follow these instructions.
have fun!
At startup time, if ovmd is enabled, it will start executing configuration scripts that need input to configure and continue. It is possible to send this configuration data through the virtual console of the VM or through the Oracle VM API. To use the Oracle VM API to send configuration messages, you have two options :
(1) use the Oracle VM CLI. As of Oracle VM 3.1, we include an Oracle VM CLI server by default when installing Oracle VM Manager. This process starts on port 10000 on the Oracle VM Manager node and acts as an ssh server. You can log into this cli using the admin username/password and then execute cli commands.
# ssh admin@localhost -p 10000 admin@localhost's password: OVM> sendVmMessage Vm name=ol6u3apitest key=foo message=bar log=no Command: sendVmMessage Vm name=ol6u3apitest key=foo message=bar log=no Status: Success Time: 2012-12-27 09:04:29,890 PST
The cli command for sending a message is sendVmMessage Vm name=[vmname] key=[key] message=[value]
If you do not want to log the out of the commands then add log=no
(2) use the Oracle VM utilities. If you install the Oracle VM Utilities, see here to get started, then :
# ./ovm_vmmessage -u admin -p ######## -h localhost -v ol6u3apitest -k foo -V bar Oracle VM VM Message utility 0.5.2. Connected. VM : 'ol6u3apitest' has status : Running. Sending message. Message sent successfully.
The ovm_vmmessage command connects to Oracle VM Manager and sends a key/value pair to the VM you select.
ovm_vmmessage -u [adminuser] -p [adminpassword] -h [managernode] -v [vmname] -k [key] -V [value]
These two commands basically allow the admin user to send simple key - value pair messages to a given VM. This is the basic mechanism we rely on to remotely configure a VM using the Oracle VM template config scripts.
For the template configuration we provide, and depending on the scripts you installed, there is a well-defined set of variables (keys) that you can set, listed below. In our scripts we have one variable that is required and this has to be set/send at the end of the configuration. This is configuring the root password. Everything else is optional. Sending the root password variable triggers the reconfiguration to execute. As an example, if you install the ovm-template-config-selinux package, then part of the configuration can be to set the selinux mode. The variable is com.oracle.linux.selinux.mode and the values can be enforcing,permissive or disabled. So to set the value of SELinux, you basically send a message with key com.oracle.linux.selinux.mode and value enforcing (or so..).
# ./ovm_vmmessage -u admin -p ######## -h localhost -v ol6u3apitest \ -k com.oracle.linux.selinux.mode -V enforcing
Do this for every variable you want to define and at the end send the root password.
# ./ovm_vmmessage -u admin -p ######## -h localhost -v ol6u3apitest \ -k com.oracle.linux.root-password -V "mypassword"
Once the above message gets sent, the ovm-template-config scripts will set up all the values and the VM will end up in a configured state. You can use this to send ssh keys, set up extra users, configure the virtual network devices etc.. To get the list of configuration variables just run # ovm-template-config --human-readable --enumerate configure and it will list the variables with a description like below.
It is also possible to selectively enable and disable scripts. This work very similar to chk-config. # ovm-chkconfig --list will show which scripts/modules are registered and whether they are enabled to run at configure time and/or cleanup time. At this point, the other options are not implemented (suspend/resume/..). If you have installed datetime but do not want to have it run or be an option, then a simple # ovm-chkconfig --target configure datetime off will disable it. This allows you, for each VM or template, to selectively enable or disable configuration options. If you disable a module then the output of ovm-template-config will reflect those changes.
The next blog entry will talk about how to make generic use of the VM message API and possible extend the ovm-template-configure modules for your own applications.
[('30', 'selinux', [{u'description': u'SELinux mode: enforcing, permissive or disabled.', u'hidden': True, u'key': u'com.oracle.linux.selinux.mode'}]), ('41', 'firewall', [{u'description': u'Whether to enable network firewall: True or False.', u'hidden': True, u'key': u'com.oracle.linux.network.firewall'}]), ('50', 'datetime', [{u'description': u'System date and time in format year-month-day-hour-minute-second, e.g., "2011-4-7-9-2-42".', u'hidden': True, u'key': u'com.oracle.linux.datetime.datetime'}, {u'description': u'System time zone, e.g., "America/New_York".', u'hidden': True, u'key': u'com.oracle.linux.datetime.timezone'}, {u'description': u'Whether to keep hardware clock in UTC: True or False.', u'hidden': True, u'key': u'com.oracle.linux.datetime.utc'}, {u'description': u'Whether to enable NTP service: True or False.', u'hidden': True, u'key': u'com.oracle.linux.datetime.ntp'}, {u'description': u'NTP servers separated by comma, e.g., "time.example.com,0.example.pool.ntp.org".', u'hidden': True, u'key': u'com.oracle.linux.datetime.ntp-servers'}, {u'description': u'Whether to enable NTP local time source: True or False.', u'hidden': True, u'key': u'com.oracle.linux.datetime.ntp-local-time-source'}]), ('50', 'network', [{u'description': u'System host name, e.g., "localhost.localdomain".', u'key': u'com.oracle.linux.network.hostname'}, {u'description': u'Hostname entry for /etc/hosts, e.g., "127.0.0.1 localhost.localdomain localhost".', u'hidden': True, u'key': u'com.oracle.linux.network.host.0'}, {u'description': u'Network device to configure, e.g., "eth0".', u'key': u'com.oracle.linux.network.device.0'}, {u'depends': u'com.oracle.linux.network.device.0', u'description': u'Network device hardware address, e.g., "00:16:3E:28:0F:4E".', u'hidden': True, u'key': u'com.oracle.linux.network.hwaddr.0'}, {u'depends': u'com.oracle.linux.network.device.0', u'description': u'Network device MTU, e.g., "1500".', u'hidden': True, u'key': u'com.oracle.linux.network.mtu.0'}, {u'choices': [u'yes', u'no'], u'depends': u'com.oracle.linux.network.device.0', u'description': u'Activate interface on system boot: yes or no.', u'key': u'com.oracle.linux.network.onboot.0'}, {u'choices': [u'dhcp', u'static'], u'depends': u'com.oracle.linux.network.device.0', u'description': u'Boot protocol: dhcp or static.', u'key': u'com.oracle.linux.network.bootproto.0'}, {u'depends': u'com.oracle.linux.network.bootproto.0', u'description': u'IP address of the interface.', u'key': u'com.oracle.linux.network.ipaddr.0', u'requires': [u'com.oracle.linux.network.bootproto.0', [u'static', u'none', None]]}, {u'depends': u'com.oracle.linux.network.bootproto.0', u'description': u'Netmask of the interface.', u'key': u'com.oracle.linux.network.netmask.0', u'requires': [u'com.oracle.linux.network.bootproto.0', [u'static', u'none', None]]}, {u'depends': u'com.oracle.linux.network.bootproto.0', u'description': u'Gateway IP address.', u'key': u'com.oracle.linux.network.gateway.0', u'requires': [u'com.oracle.linux.network.bootproto.0', [u'static', u'none', None]]}, {u'depends': u'com.oracle.linux.network.bootproto.0', u'description': u'DNS servers separated by comma, e.g., "220.127.116.11,18.104.22.168".', u'key': u'com.oracle.linux.network.dns-servers.0', u'requires': [u'com.oracle.linux.network.bootproto.0', [u'static', u'none', None]]}, {u'description': u'DNS search domains separated by comma, e.g., "us.example.com,cn.example.com".', u'hidden': True, u'key': u'com.oracle.linux.network.dns-search-domains.0'}, {u'description': u'Network device to configure, e.g., "eth0".', u'hidden': True, u'key': u'com.oracle.linux.network.device.1'}, {u'depends': u'com.oracle.linux.network.device.1', u'description': u'Network device hardware address, e.g., "00:16:3E:28:0F:4E".', u'hidden': True, u'key': u'com.oracle.linux.network.hwaddr.1'}, {u'depends': u'com.oracle.linux.network.device.1', u'description': u'Network device MTU, e.g., "1500".', u'hidden': True, u'key': u'com.oracle.linux.network.mtu.1'}, {u'choices': [u'yes', u'no'], u'depends': u'com.oracle.linux.network.device.1', u'description': u'Activate interface on system boot: yes or no.', u'hidden': True, u'key': u'com.oracle.linux.network.onboot.1'}, {u'choices': [u'dhcp', u'static'], u'depends': u'com.oracle.linux.network.device.1', u'description': u'Boot protocol: dhcp or static.', u'hidden': True, u'key': u'com.oracle.linux.network.bootproto.1'}, {u'depends': u'com.oracle.linux.network.bootproto.1', u'description': u'IP address of the interface.', u'hidden': True, u'key': u'com.oracle.linux.network.ipaddr.1', u'requires': [u'com.oracle.linux.network.bootproto.1', [u'static', u'none', None]]}, {u'depends': u'com.oracle.linux.network.bootproto.1', u'description': u'Netmask of the interface.', u'hidden': True, u'key': u'com.oracle.linux.network.netmask.1', u'requires': [u'com.oracle.linux.network.bootproto.1', [u'static', u'none', None]]}, {u'depends': u'com.oracle.linux.network.bootproto.1', u'description': u'Gateway IP address.', u'hidden': True, u'key': u'com.oracle.linux.network.gateway.1', u'requires': [u'com.oracle.linux.network.bootproto.1', [u'static', u'none', None]]}, {u'depends': u'com.oracle.linux.network.bootproto.1', u'description': u'DNS servers separated by comma, e.g., "126.96.36.199,126.96.36.199".', u'hidden': True, u'key': u'com.oracle.linux.network.dns-servers.1', u'requires': [u'com.oracle.linux.network.bootproto.1', [u'static', u'none', None]]}, {u'description': u'DNS search domains separated by comma, e.g., "us.example.com,cn.example.com".', u'hidden': True, u'key': u'com.oracle.linux.network.dns-search-domains.1'}]), ('60', 'user', [{u'description': u'Name of the user on which to perform operation.', u'hidden': True, u'key': u'com.oracle.linux.user.name.0'}, {u'description': u'Action to perform on the user: add, del or mod.', u'hidden': True, u'key': u'com.oracle.linux.user.action.0'}, {u'description': u'User ID.', u'hidden': True, u'key': u'com.oracle.linux.user.uid.0'}, {u'description': u'User initial login group.', u'hidden': True, u'key': u'com.oracle.linux.user.group.0'}, {u'description': u'Supplementary groups separated by comma.', u'hidden': True, u'key': u'com.oracle.linux.user.groups.0'}, {u'description': u'User password.', u'hidden': True, u'key': u'com.oracle.linux.user.password.0', u'password': True}, {u'description': u'New name of the user.', u'hidden': True, u'key': u'com.oracle.linux.user.new-name.0'}, {u'description': u'Name of the group on which to perform operation.', u'hidden': True, u'key': u'com.oracle.linux.group.name.0'}, {u'description': u'Action to perform on the group: add, del or mod.', u'hidden': True, u'key': u'com.oracle.linux.group.action.0'}, {u'description': u'Group ID.', u'hidden': True, u'key': u'com.oracle.linux.group.gid.0'}, {u'description': u'New name of the group.', u'hidden': True, u'key': u'com.oracle.linux.group.new-name.0'}]), ('70', 'ssh', [{u'description': u'Host private rsa1 key for protocol version 1.', u'hidden': True, u'key': u'com.oracle.linux.ssh.host-key'}, {u'description': u'Host public rsa1 key for protocol version 1.', u'hidden': True, u'key': u'com.oracle.linux.ssh.host-key-pub'}, {u'description': u'Host private rsa key.', u'hidden': True, u'key': u'com.oracle.linux.ssh.host-rsa-key'}, {u'description': u'Host public rsa key.', u'hidden': True, u'key': u'com.oracle.linux.ssh.host-rsa-key-pub'}, {u'description': u'Host private dsa key.', u'hidden': True, u'key': u'com.oracle.linux.ssh.host-dsa-key'}, {u'description': u'Host public dsa key.', u'hidden': True, u'key': u'com.oracle.linux.ssh.host-dsa-key-pub'}, {u'description': u'Name of the user to add a key.', u'hidden': True, u'key': u'com.oracle.linux.ssh.user.0'}, {u'description': u'Authorized public keys.', u'hidden': True, u'key': u'com.oracle.linux.ssh.authorized-keys.0'}, {u'description': u'Private key for authentication.', u'hidden': True, u'key': u'com.oracle.linux.ssh.private-key.0'}, {u'description': u'Private key type: rsa, dsa or rsa1.', u'hidden': True, u'key': u'com.oracle.linux.ssh.private-key-type.0'}, {u'description': u'Known hosts.', u'hidden': True, u'key': u'com.oracle.linux.ssh.known-hosts.0'}]), ('90', 'authentication', [{u'description': u'System root password.', u'key': u'com.oracle.linux.root-password', u'password': True, u'required': True}])]
Oracle Linux 6.3 comes with kernel 2.6.39-200.24.1 (UEK2)
# uname -a Linux ol6u3 2.6.39-200.24.1.el6uek.x86_64 #1 SMP Sat Jun 23 02:39:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
Oracle Linux updates are freely available on our public-yum site and the default install of Oracle Linux 6.3 already points to this location for updates.
# yum update # reboot # uname -a Linux ol6u3 2.6.39-300.17.3.el6uek.x86_64 #1 SMP Wed Dec 19 06:28:03 PST 2012 x86_64 x86_64 x86_64 GNU/Linux
There is an extra kernel module required for the Oracle VM API to work, the ovmapi kernel module provides the ability to communicate messages back and forth between the host and the VM and as such between Oracle VM Manager, through the VM API to the VM and back. We included this kernel module in the 2.6.39-300 kernel to make it easy. There is no need to install extra kernel modules or keep kernel modules up to date when or if we have a new update. The source code for this kernel module is of course part of the UEK2 source tree.
After reboot, download the latest public-yum repo file from public-yum which contains more repositories and enable the add-on channel which contains the Oracle VM API packages:
inside the VM :
# cd /etc/yum.repos.d # rm public-yum-ol6.repo <- (replace the original version with this newer version) # wget http://public-yum.oracle.com/public-yum-ol6.repo
Find the ol6_addons section and change enabled=0 to enabled=1.
[ol6_addons] name=Oracle Linux $releasever Add ons ($basearch) baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL6/addons/$basearch/ gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6 gpgcheck=1 enabled=1
Save the file.
# yum install ovmd xenstoreprovider python-simplejson ovm-template-config
This installs the basic necessary packages on Oracle Linux 6 to support the Oracle VM API. xenstore provider is the library which communicates with the ovmapi kernel infrastructure. ovmd is a daemon that handles configuration and re-configuration events and provides a mechanism to send/receive messages between the VM and the Oracle VM Manager.
In order to be able to create a VM template that includes basic OS configuration system scripts, you can decide to install any or all of the following :
ovm-template-config-authentication : Oracle VM template authentication configuration script. ovm-template-config-datetime : Oracle VM template datetime configuration script. ovm-template-config-firewall : Oracle VM template firewall configuration script. ovm-template-config-network : Oracle VM template network configuration script. ovm-template-config-selinux : Oracle VM template selinux configuration script. ovm-template-config-ssh : Oracle VM template ssh configuration script. ovm-template-config-system : Oracle VM template system configuration script. ovm-template-config-user : Oracle VM template user configuration script.
Simply type # yum install ovm-template-config-... to install whichever you want.
To enable ovmd (recommended) do :
# chkconfig ovmd on # /etc/init.d/ovmd start
If you want to shutdown this VM and enable the first boot configuration as a template, execute :
# ovmd -s cleanup # service ovmd enable-initial-config # shutdown -h now
After cloning this VM or starting it, it will act as a first time boot VM and it will require configuration input through the VM API or on the virtual VM console.
My next blog will go into detail on how to send messages through the Oracle VM API for remote configuration and also how to extend the scripts.
To show you how easy and powerful this is, I took some of the Oracle VM agent Python code, this is a very simple example of how to create a lock domain, a lock and when you know you get the lock or not. The focus here is just a master lock which y ou could use for an agent that is responsible for a virtual IP or some executable that you want to locate on a given server but the calls to create any kind of lock are in the code. Anyone that wants to experiment with this can add their own bits in a matter of minutes.
The prerequisite is simple : take a number of servers, configure an ocfs2 volume and an ocfs2 cluster (see previous blog entries) and run the script. You do not have to set up an ocfs2 volume if you do not want to, you could just set up the domain without actually mounting the filesystem. (See the global heartbeat blog). So practically this can be done with a very small simple setup.
My example has two nodes, wcoekaer-emgc1 and wcoekaer-emgc2 are the two Oracle Linux 6 nodes, configured with a shared disk and an ocfs2 filesystem mounted. This setup ensures that the dlmfs kernel module is loaded and the cluster is online. Take the python code listed here and just execute it on both nodes.
[root@wcoekaer-emgc2 ~]# lsmod |grep ocfs ocfs2 1092529 1 ocfs2_dlmfs 20160 1 ocfs2_stack_o2cb 4103 1 ocfs2_dlm 228380 1 ocfs2_stack_o2cb ocfs2_nodemanager 219951 12 ocfs2,ocfs2_dlmfs,ocfs2_stack_o2cb,ocfs2_dlm ocfs2_stackglue 11896 3 ocfs2,ocfs2_dlmfs,ocfs2_stack_o2cb configfs 29244 2 ocfs2_nodemanager jbd2 93114 2 ocfs2,ext4You see that the ocfs2_dlmfs kernel module is loaded.
[root@wcoekaer-emgc2 ~]# mount |grep dlmfs ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw)The dlmfs virtual filesystem is mounted on /dlm.
I now execute dlm.py on both nodes and show some output, after a while I kill (control-c) the script on the master node and you see the other node take over the lock. I then restart the dlm.py script and reboot the other node and you see the same.
[root@wcoekaer-emgc1 ~]# ./dlm.py Checking DLM DLM Ready - joining domain : mycluster Starting main loop... i am master of the multiverse i am master of the multiverse i am master of the multiverse i am master of the multiverse ^Ccleaned up master lock file [root@wcoekaer-emgc1 ~]# ./dlm.py Checking DLM DLM Ready - joining domain : mycluster Starting main loop... i am not the master i am not the master i am not the master i am not the master i am master of the multiverseThis shows that I started as master, then hit ctrl-c, I drop the lock, the other node takes the lock, then I reboot the other node and I take the lock again.
[root@wcoekaer-emgc2 ~]# ./dlm.py Checking DLM DLM Ready - joining domain : mycluster Starting main loop... i am not the master i am not the master i am not the master i am not the master i am master of the multiverse i am master of the multiverse i am master of the multiverse ^Z [1]+ Stopped ./dlm.py [root@wcoekaer-emgc2 ~]# bg [1]+ ./dlm.py & [root@wcoekaer-emgc2 ~]# reboot -fHere you see that when this node started without being master, then at time of ctrl-c on the other node, became master, then after a forced reboot, the lock automatically gets released.
And here is the code, just copy it to your servers and execute it...
#!/usr/bin/python # Copyright (C) 2006-2012 Oracle. All rights reserved. # # This program is free software; you can redistribute it and/or modify it under # the terms of the GNU General Public License as published by the Free Software # Foundation, version 2. This program is distributed in the hope that it will # be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General # Public License for more details. You should have received a copy of the GNU # General Public License along with this program; if not, write to the Free # Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA # 021110-1307, USA. import sys import subprocess import stat import time import os import re import socket from time import sleep from os.path import join, isdir, exists # defines # dlmfs is where the dlmfs filesystem is mounted # the default, normal place for ocfs2 setups is /dlm # ocfs2_dlmfs on /dlm type ocfs2_dlmfs (rw) DLMFS = "/dlm" # we need a domain name which really is just a subdir in dlmfs # default to "mycluster" so then it creates /dlm/mycluster # locks are created inside this directory/domain DLM_DOMAIN_NAME = "mycluster" DLM_DOMAIN_PATH = DLMFS + "/" + DLM_DOMAIN_NAME # the main lock to use for being the owner of a lock # this can be any name, the filename is just the lockname DLM_LOCK_MASTER = DLM_DOMAIN_PATH + "/" + "master" # just a timeout SLEEP_ON_ERR = 60 def run_cmd(cmd, success_return_code=(0,)): if not isinstance(cmd, list): raise Exception("Only accepts list!") cmd = [str(x) for x in cmd] proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) (stdoutdata, stderrdata) = proc.communicate() if proc.returncode not in success_return_code: raise RuntimeError('Command: %s failed (%s): stderr: %s stdout: %s' % (cmd, proc.returncode, stderrdata, stdoutdata)) return str(stdoutdata) def dlm_ready(): """ Indicate if the DLM is ready of not. With dlmfs, the DLM is ready once the DLM filesystem is mounted under /dlm. @return: C{True} if the DLM is ready, C{False} otherwise. @rtype: C{bool} """ return os.path.ismount(DLMFS) # just do a mkdir, if it already exists, we're good, if not just create it def dlm_join_domain(domain=DLM_DOMAIN_NAME): _dir = join(DLMFS, domain) if not isdir(_dir): os.mkdir(_dir) # else: already joined # leaving a domain is basically removing the directory. def dlm_leave_domain(domain=DLM_DOMAIN_NAME, force=True): _dir = join(DLMFS, domain) if force: cmd = ["rm", "-fr", _dir] else: cmd = ["rmdir", _dir] run_cmd(cmd) # acquire a lock def dlm_acquire_lock(lock): dlm_join_domain() # a lock is a filename in the domain directory lock_path = join(DLM_DOMAIN_PATH, lock) try: if not exists(lock_path): fd = os.open(lock_path, os.O_CREAT | os.O_NONBLOCK) os.close(fd) # create the EX lock # creating a file with O_RDWR causes an EX lock fd = os.open(lock_path, os.O_RDWR | os.O_NONBLOCK) # once the file is created in this mode, you can close it # and you still keep the lock os.close(fd) except Exception, e: if exists(lock_path): os.remove(lock_path) raise e def dlm_release_lock(lock): # releasing a lock is as easy as just removing the file lock_path = join(DLM_DOMAIN_PATH, lock) if exists(lock_path): os.remove(lock_path) def acquire_master_dlm_lock(): ETXTBUSY = 26 dlm_join_domain() # close() does not downconvert the lock level nor does it drop the lock. The # holder still owns the lock at that level after close. # close() allows any downconvert request to succeed. # However, a downconvert request is only generated for queued requests. And # O_NONBLOCK is specifically a noqueue dlm request. # 1) O_CREAT | O_NONBLOCK will create a lock file if it does not exist, whether # we are the lock holder or not. # 2) if we hold O_RDWR lock, and we close but not delete it, we still hold it. # afterward, O_RDWR will succeed, but O_RDWR | O_NONBLOCK will not. # 3) but if we donnot hold the lock, O_RDWR will hang there waiting, # which is not desirable -- any uninterruptable hang is undesirable. # 4) if noboday else hold the lock either, but the lock file exists as side effect # of 1), with O_NONBLOCK, it may result in ETXTBUSY # a) we need O_NONBLOCK to avoid scenario (3) # b) we need to delete it ourself to avoid (2) # *) if we do not succeed with (1), remove the lock file to avoid (4) # *) if everything is good, we drop it and we remove it # *) if killed by a program, this program should remove the file # *) if crashed, but not rebooted, something needs to remove the file # *) on reboot/reset the lock is released to the other node(s) try: if not exists(DLM_LOCK_MASTER): fd = os.open(DLM_LOCK_MASTER, os.O_CREAT | os.O_NONBLOCK) os.close(fd) master_lock = os.open(DLM_LOCK_MASTER, os.O_RDWR | os.O_NONBLOCK) os.close(master_lock) print "i am master of the multiverse" # at this point, I know I am the master and I can add code to do # things that only a master can do, such as, consider setting # a virtual IP or, if I am master, I start a program # and if not, then I make sure I don't run that program (or VIP) # so the magic starts here... return True except OSError, e: if e.errno == ETXTBUSY: print "i am not the master" # if we are not master and the file exists, remove it or # we will never succeed if exists(DLM_LOCK_MASTER): os.remove(DLM_LOCK_MASTER) else: raise e def release_master_dlm_lock(): if exists(DLM_LOCK_MASTER): os.remove(DLM_LOCK_MASTER) def run_forever(): # set socket default timeout for all connections print "Checking DLM" if dlm_ready(): print "DLM Ready - joining domain : " + DLM_DOMAIN_NAME else: print "DLM not ready - bailing, fix the cluster stack" sys.exit(1) dlm_join_domain() print "Starting main loop..." socket.setdefaulttimeout(30) while True: try: acquire_master_dlm_lock() sleep(20) except Exception, e: sleep(SLEEP_ON_ERR) except (KeyboardInterrupt, SystemExit): if exists(DLM_LOCK_MASTER): os.remove(DLM_LOCK_MASTER) # if you control-c out of this, then you lose the lock! # delete it on exit for release print "cleaned up master lock file" sys.exit(1) if __name__ == '__main__': run_forever()
One of the main features added most recently is Global Heartbeat. OCFS2 as a filesystem typically was used with what's called local heartbeat. Basically for every filesystem you mounted, it would start its own local heartbeat, membership mechanism. The disk heartbeat means a disk io every 1 or 2 seconds for every node in the cluster, for every device. It was never a problem when the number of mounted volumes was relatively small but once customers were using 20+ volumes the overhead of the multiple disk heartbeats became significant and at times became a stability issue.
global heartbeat was written to provide a solution to the multiple heartbeats. It is now possible to specify on which device(s) you want a heartbeat thread and then you can mount many other volumes that do not have their own and the heartbeat is shared amongst those one, or few threads and as such significantly reducing disk IO overhead.
I was playing with this a little bit the other day and noticed that this wasn't very well documented so why not write it up here and share it with everyone. Getting started with OCFS2 is just really easy and withing just a few minutes it is possible to have a complete installation.
I started with two servers installed with Oracle Linux 6.3. Each server has 2 network interfaces, one public and one private. The servers have a local disk and a shared storage device. For cluster filesystems, typically this shared storage device should be either a shared SAN disk or an iscsi device but it is also possible with Oracle Linux and UEK2 to create a shared virtual device on an nfs server and use this device for the cluster filesystem. This technique is used with Oracle VM where the shared storage is NAS-based.I just wrote a blog entry about how to do that here.
While it is technically possible to create a working ocfs2 configuration using just one network and a single IP per server, it is certainly not ideal and not a recommended configuration for real world use. In any cluster environment it's highly recommended to have a private network for cluster traffic.The biggest reason for instability in a clustering environment is a bad/unreliable network and/or storage. Many times the environment has an overloaded network which causes network heartbeats to fail or disks where failover takes longer than the default configuration and the only alternative we have at that point, is to reboot the node(s).
Typically when I do a test like this, I make sure I use the latest versions of the OS release. So after an installation of Oracle Linux 6.3, I just do a yum update on all my nodes to have the latest packages and also latest kernel version installed and then do a reboot. That gets me to 2.6.39-300.17.3.el6uek.x86_64 at the time of writing. Of course all this is freely accessibly from http://public-yum.oracle.com.
Depending on the type of installation you did (basic, minimal, etc...) you may or may not have to add RPMs. Do a simple check rpm -q ocfs2-tools to see if the tools are installed, if not, just run yum install ocfs2-tools. And that's it. All required software is now installed. The kernel modules are already part of the uek2 kernel and the required tools (mkfs, fsck, o2cb,..) are part of the ocfs2-tools RPM.
Next up: create the filesystem on the shared disk device and configure the cluster.
One requirement for using global heartbeat is that the heartbeat device needs to be a NON-partitioned disk. Other OCFS2 volumes you want to create and mount can be on partitioned disks, but a device for the heartbeat needs to be on an empty disk. Let's assume /dev/sdb in this example.
# mkfs.ocfs2 -b 4K -C 4K -J size=4M -N 4 -L ocfs2vol1 \ --cluster-name=ocfs2 --cluster-stack=o2cb --global-heartbeat /dev/sdbThis creates a filesystem with a 4K blocksize (normal value), clustersize of 4K (if you have many small files, this is a good value, if you have few large files, go to 1M).
Journalsize of 4M if you have a large filesystem with a lot of metadata changes you might want to increase this. I did not add an option for 32bit or 64bit journals. if you want to create huge filesystems then use block64 which uses jbd2.
The filesystem is created for 4 nodes (-N 4) this can be modified if your cluster needs to grow larger so you can always tune this with tunefs.ocfs2.
Label ocfs2vol1, this is a disklabel you can later use to mount by label a filesystem.
clustername=ocfs2, this is the default name but if you want to have your own name for your cluster you can put a different value here, remember it because you will need to configure the clusterstack with the clustername later.
cluster-stack=o2cb : it is possible to have different cluster-stacks used such as pacemaker or cman.
global-heartbeat : make sure that the filesystem is prepared and built to support global heartbeat
/dev/sdb : the device to use for the filesystem.
# mkfs.ocfs2 -b 4K -C 4K -J size=4M -N 4 -L ocfs2vol1 --cluster-name=ocfs2 \ --cluster-stack=o2cb --force --global-heartbeat /dev/sdb mkfs.ocfs2 1.8.0 Cluster stack: o2cb Cluster name: ocfs2 Stack Flags: 0x1 NOTE: Feature extended slot map may be enabled Overwriting existing ocfs2 partition. WARNING: Cluster check disabled. Proceed (y/N): y Label: ocfs2vol1 Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg Block size: 4096 (12 bits) Cluster size: 4096 (12 bits) Volume size: 10725765120 (2618595 clusters) (2618595 blocks) Cluster groups: 82 (tail covers 5859 clusters, rest cover 32256 clusters) Extent allocator size: 4194304 (1 groups) Journal size: 4194304 Node slots: 4 Creating bitmaps: done Initializing superblock: done Writing system files: done Writing superblock: done Writing backup superblock: 2 block(s) Formatting Journals: done Growing extent allocator: done Formatting slot map: done Formatting quota files: done Writing lost+found: done mkfs.ocfs2 successful
Now, we just have to configure the o2cb stack and we're done.
# mounted.ocfs2 -d Device Stack Cluster F UUID Label /dev/sdb o2cb ocfs2 G 244A6AAAE77F4053803734530FC4E0B7 ocfs2vol1o2cb add-heartbeat ocfs2 244A6AAAE77F4053803734530FC4E0B7
That's it. If you want to enable this at boot time, you can configure o2cb to start automatically by running /etc/init.d/o2cb configure. This allows you to set different heartbeat time out values and also whether or not to start the clusterstack at boot time.
Now that a first node is configured, all you have to do is copy the file /etc/ocfs2/cluster.conf to all the other nodes in your cluster. You do not have to edit it on the other nodes, you just need to have an exact copy everywhere. You also do not need to redo the above commands, except 1) make sure ocfs2-tools is installed everywhere and if you want to start at boot time, re-run the /etc/init.d/o2cb configure on the other nodes as well. From here on, you can just mount your filesystems :
mount /dev/sdb /mountpoint1 on each node.
If you create more OCFS2 volumes you can just keep mounting them all, and with global heartbeat, you will just have one (or a few) hb's going on.
have fun...
Here is vmstat output, the first output shows a single heartbeat and 8 mounted filesystems, the second vmstat output shows 8 mounted filesystems with their own local heartbeat. Even though the IO amount is low, it shows that there are about 8x more IOs happening (from 1 every other second to 4 every second). As these are small IOs, they will move the diskhead to a specific place all the time and interrupt performance if you have it on each device. Hopefully this shows the benefits of global heartbeat.
# vmstat 1 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 789752 26220 97620 0 0 1 0 41 34 0 0 100 0 0 0 0 0 789752 26220 97620 0 0 0 0 46 22 0 0 100 0 0 0 0 0 789752 26220 97620 0 0 1 1 38 29 0 0 100 0 0 0 0 0 789752 26228 97620 0 0 0 52 52 41 0 0 100 1 0 0 0 0 789752 26228 97620 0 0 1 0 28 26 0 0 100 0 0 0 0 0 789760 26228 97620 0 0 0 0 30 30 0 0 100 0 0 0 0 0 789760 26228 97620 0 0 1 1 26 20 0 0 100 0 0 0 0 0 789760 26228 97620 0 0 0 0 54 37 0 1 100 0 0 0 0 0 789760 26228 97620 0 0 1 0 29 28 0 0 100 0 0 0 0 0 789760 26236 97612 0 0 0 16 43 48 0 0 100 0 0 0 0 0 789760 26236 97620 0 0 1 1 48 28 0 0 100 0 0 0 0 0 789760 26236 97620 0 0 0 0 42 30 0 0 100 0 0 0 0 0 789760 26236 97620 0 0 1 0 26 30 0 0 100 0 0 0 0 0 789760 26236 97620 0 0 0 0 35 24 0 0 100 0 0 0 1 0 789760 26240 97616 0 0 1 21 29 27 0 0 100 0 0 0 0 0 789760 26244 97620 0 0 0 4 51 44 0 0 100 0 0 0 0 0 789760 26244 97620 0 0 1 0 31 24 0 0 100 0 0 0 0 0 789760 26244 97620 0 0 0 0 25 28 0 0 100 0 0 0 0 0 789760 26244 97620 0 0 1 1 30 20 0 0 100 0 0 0 0 0 789760 26244 97620 0 0 0 0 41 30 0 0 100 0 0 0 0 0 789760 26252 97616 0 0 1 16 56 44 0 0 100 0 0 # vmstat 1 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 784364 28732 98620 0 0 4 46 54 64 0 0 100 0 0 0 0 0 784364 28732 98628 0 0 4 2 60 48 0 0 100 0 0 0 0 0 784364 28732 98628 0 0 4 2 51 53 0 0 100 0 0 0 0 0 784364 28732 98628 0 0 4 2 58 50 0 0 100 0 0 0 0 0 784364 28732 98628 0 0 4 2 56 44 0 0 100 0 0 0 0 0 784364 28732 98628 0 0 4 2 46 47 0 0 100 0 0 0 0 0 784364 28732 98628 0 0 4 2 65 54 0 0 100 0 0 0 0 0 784388 28740 98620 0 0 4 14 65 55 0 0 100 0 0 0 0 0 784388 28740 98628 0 0 4 2 46 48 0 0 100 0 0 0 0 0 784388 28740 98628 0 0 4 2 52 42 0 0 100 0 0 0 0 0 784388 28740 98628 0 0 4 2 51 58 0 0 100 0 0 0 0 0 784388 28740 98628 0 0 4 2 36 43 0 0 100 0 0 0 0 0 784396 28740 98628 0 0 4 2 39 47 0 0 100 0 0 0 0 0 784396 28740 98628 0 0 4 2 52 54 0 0 100 0 0 0 0 0 784396 28740 98628 0 0 4 2 42 48 0 0 100 0 0 0 0 0 784404 28748 98620 0 0 4 14 52 63 0 0 100 0 0 0 0 0 784404 28748 98628 0 0 4 2 32 42 0 0 100 0 0 0 0 0 784404 28748 98628 0 0 4 2 50 40 0 0 100 0 0 0 0 0 784404 28748 98628 0 0 4 2 58 56 0 0 100 0 0 0 0 0 784412 28748 98628 0 0 4 2 39 46 0 0 100 0 0 0 0 0 784412 28748 98628 0 0 4 2 45 50 0 0 100 0 0 0 0 0 784412 28748 98628 0 0 4 2 43 42 0 0 100 0 0 0 0 0 784288 28748 98628 0 0 4 6 48 52 0 0 100 0 0
Oracle VM clustering relies on the OCFS2 clusterstack/filesystem that is native in the kernel (uek2/2.6.39-x). When we create an HA-enabled pool, we create, what we call, a pool filesystem. That filesystem contains an ocfs2 volume so that we can store cluster-wide data. In particular we store shared database files that are needed by the Oracle VM agents on the nodes for HA. It contains info on pool membership, which VMs are in HA mode, what the pool IP is etc...
When the user provides an nfs filesystem for the pool, we do the following :
If someone wants to try out something that relies on block-based shared storage devices, such as ocfs2, but does not have iSCSI or SAN storage, using nfs is an alternative and dm nfs just makes it really easy.
To do this yourself, the following commands will do it for you :
mount mynfsserver:/mountpoint /mnt dd if=/dev/zero of=/mnt/myvolume.img bs=1M count=2000 dmsetup create myvolume --table "0 4096000 nfs /mnt/myvolume.img 0"So mount the nfs volume, create a file which will be the container of the blockdevice, in this case a 2GB file and then create the dm device. The values for the dmsetup command are the following:
myvolume = the name of the /dev/mapper device. Here we end up with /dev/mapper/myvolume
table = start (normally always 0), number of blocks/length, this is in 512byte blocks, so you double the number,nfs since this is on nfs, filename of the nfs based file, offset (normally always 0)
So now you have /dev/mapper/myvolume, it acts like a normal block device. If you do this on multiple servers, you can actually create an ocfs2 filesystem on this block device and it will be consistent across the servers.
Credits go to Chuck Lever for writing dm nfs in the first place, thanks Chuck :) The code for dm nfs is here.
When a new stable mainline kernel is released by Linus or GregKH, we internally build RPMs to test it and do some QA work around it to keep track of what's going on with the latest development kernels. It helps us understand how performance moves up or down and if there are issues, we try to help look into them and of course send that stuff back upstream. Many Linux users out there are interested in trying out the latest features but there are some potential barriers to do this.
(1) in general, you are looking at an upstream development distribution, which means that everything changes both in userspace(random applications) and kernel. Projects like Fedora are very useful and someone that wants to just see how the entire distribution evolves with all the changes, this is a great way to be current. A drawback here, though, is that if you have applications that are not part of the distribution, there's a lot of manual work involved or they might just not work because the changes are too drastic. The introduction of systemd is a good example.
(2) when you look at many of our customers, that are interested in our database products or applications, the starting point of having a supported/certified userspace/distribution, like Oracle Linux, is a much easier way to get your feet wet in seeing what new/future Linux kernel enhancements could do.
This is where the playground channel comes into play. When you install Oracle Linux 6 (which anyone can download and use from http://edelivery.oracle.com/linux), grab the latest public yum repository file http://public-yum.oracle.com/public-yum-ol6.repo, put it in /etc/yum.repos.d and enable the playground repo :
[ol6_playground_latest] name=Latest mainline stable kernel for Oracle Linux 6 ($basearch) - Unsupported baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL6/playground/latest/$basearch/ gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6 gpgcheck=1 enabled=1Now, all you need to do : type yum update and you will be downloading the latest stable kernel which will install cleanly on Oracle Linux 6. Thus you end up with a stable Linux distribution where you can install all your software, and then download the latest stable kernel (at time of writing this is 3.6.7) without having to recompile a kernel, without having to jump through hoops.
There is of course a big, very important disclaimer this is NOT for PRODUCTION use.
We want to try and help make it easy for people that are interested, from a user perspective, where the Linux kernel is going and make it easy to install and use it and play around with new features. Without having to learn how to compile a kernel and without necessarily having to install a complete new distribution with all the changes top to bottom.
So we don't or won't introduce any new userspace changes, this project really is around making it easy to try out the latest upstream Linux kernels in a very easy way on an environment that's stable and you can keep current, since all the latest errata for Oracle Linux 6 are published on the public yum repo as well. So one repository location for all your current changes and the upstream kernels. We hope that this will get more users to try out the latest kernel and report their findings. We are always interested in understanding stability and performance characteristics.
As new features are going into the mainline kernel, that could potentially be interesting or useful for various products, we will try to point them out on our blogs and give an example on how something can be used so you can try it out for yourselves.
Anyway, I hope people will find this useful and that it will help increase interested in upstream development beyond reading lkml by some of the more non-kernel-developer types.
The VM's can be downloaded here. There are a few important things :
note : when creating the VM in VirtualBox, go into the netowrk settings of the VM and make sure that your virtual network is associated with the correct physical network. These VMs were exported on a Linux server with the virtual networks bound to eth0 as a device. On a Mac or a Windows PC this is likely a different name so to be safe just modify this before starting the VM
The Manager VM is like the previous version 3.0. The VM starts and does an auto-login into Xwindows and there is a readme file that opens up in firefox and a login button that starts firefox to the local management port for Oracle VM.
the server root password and ovs-agent password (to discover the server is ovsroot)
Oracle ASMLib was introduced at the time Oracle released Oracle Database 10g R1. 10gR1 introduced a very cool important new features called Oracle ASM (Automatic Storage Management). A very simplistic description would be that this is a very sophisticated volume manager for Oracle data. Give your devices directly to the ASM instance and we manage the storage for you, clustered, highly available, redundant, performance, etc, etc... We recommend using Oracle ASM for all database deployments, single instance or clustered (RAC).
The ASM instance manages the storage and every Oracle server process opens and operates on the storage devices like it would open and operate on regular datafiles or raw devices. So by default since 10gR1 up to today, we do not interact differently with ASM managed block devices than we did before with a datafile being mapped to a raw device. All of this is without ASMLib, so ignore that one for now. Standard Oracle on any platform that we support (Linux, Windows, Solaris, AIX, ...) does it the exact same way. You start an ASM instance, it handles storage management, all the database instances use and open that storage and read/write from/to it. There are no extra pieces of software needed, including on Linux. ASM is fully functional and selfcontained without any other components.
In order for the admin to provide a raw device to ASM or to the database, it has to have persistent device naming. If you booted up a server where a raw disk was named /dev/sdf and you give it to ASM (or even just creating a tablespace without asm on that device with datafile '/dev/sdf') and next time you boot up and that device is now /dev/sdg, you end up with an error. Just like you can't just change datafile names, you can't change device filenames without telling the database, or ASM. persistent device naming on Linux, especially back in those days ways to say it bluntly, a nightmare. In fact there were a number of issues (dating back to 2004) :
Correction to the above: ASM can handle device name changes across reboots with the correct ASM_DISKSTRING in the init.ora, it will be able to find the disks even if they changed, however part of device naming and device metadata on reboot is the correct ownership (oracle:dba ....). With ASMLib in place, this is not an issue and it will take care of ownership and permissions of the ASM disk devices.
So given the above, we tried to find a way to make this easier on the admins, in many ways, similar to why we started working on OCFS a few years earlier -> how can we make life easier for the admins on Linux.
A feature of Oracle ASM is the ability for third parties to write an extension using what's called ASMLib. It is possible for any third party OS or storage vendor to write a library using a specific Oracle defined interface that gets used by the ASM instance and by the database instance when available. This interface offered 2 components :
This is similar to a library that a number of companies have implemented over many years called libODM (Oracle Disk Manager). ODM was specified many years before we introduced ASM and allowed third party vendors to implement their own IO routines so that the database would use this library if installed and make use of the library open/read/write/close,.. routines instead of the standard OS interfaces. PolyServe back in the day used this to optimize their storage solution, Veritas used (and I believe still uses) this for their filesystem. It basically allowed, in particular, filesystem vendors to write libraries that could optimize access to their storage or filesystem.. so ASMLib was not something new, it was basically based on the same model. You have libodm for just database access, you have libasm for asm/database access.
Since this library interface existed, we decided to do a reference implementation on Linux. We wrote an ASMLib for Linux that could be used on any Linux platform and other vendors could see how this worked and potentially implement their own solution. As I mentioned earlier, ASMLib and ODMLib are libraries for third party extensions. ASMLib for Linux, since it was a reference implementation implemented both interfaces, the storage discovery part and the IO part. There are 2 components :
The userspace library is a binary-only module since it links with and contains Oracle header files but is generic, we only have one asm library for the various Linux platforms. This library is opened by Oracle ASM and by Oracle database processes and this library interacts with the OS through the asm device (/dev/asm). It can install on Oracle Linux, on SuSE SLES, on Red Hat RHEL,.. The library itself doesn't actually care much about the OS version, the kernel module and device cares. The support tools are simple scripts that allow the admin to label devices and scan for disks and devices. This way you can say create an ASM disk label foo on, currently /dev/sdf... So if /dev/sdf disappears and next time is /dev/sdg, we just scan for the label foo and we discover it as /dev/sdg and life goes on without any worry. Also, when the database needs access to the device, we don't have to worry about file permissions or anything it will be taken care of. So it's a convenience thing.
Correction: the extra advantage with ASMLib here being the fact that it will take care of the file permissions and ownership of the device.
The kernel module oracleasm.ko is a Linux kernel module/device driver. It implements a device /dev/oracleasm/* and any and all IO goes through ASMLib -> /dev/oracleasm. This kernel module is obviously a very specific Oracle related device driver but it was released under the GPL v2 so anyone could easily build it for their Linux distribution kernels.
Advantages for using ASMLib :
Just like with OCFS and OCFS2, each kernel version (major or minor) has to get a new version of the device drivers. We started out building the oracleasm kernel module rpms for many distributions, SLES (in fact in the early days still even for this thing called United Linux) and RHEL. The driver didn't make sense to get pushed into upstream Linux because it's unique and specific to the Oracle database.
As it takes a huge effort in terms of build infrastructure and QA and release management to build kernel modules for every architecture, every linux distribution and every major and minor version we worked with the vendors to get them to add this tiny kernel module to their infrastructure. (60k source code file). The folks at SuSE understood this was good for them and their customers and us and added it to SLES. So every build coming from SuSE for SLES contains the oracleasm.ko module. We weren't as successful with other vendors so for quite some time we continued to build it for RHEL and of course as we introduced Oracle Linux end of 2006 also for Oracle Linux. With Oracle Linux it became easy for us because we just added the code to our build system and as we churned out Oracle Linux kernels whether it was for a public release or for customers that needed a one off fix where they also used asmlib, we didn't have to do any extra work it was just all nicely integrated.
With the introduction of Oracle Linux's Unbreakable Enterprise Kernel and our interest in being able to exploit ASMLib more, we started working on a very exciting project called Data Integrity. Oracle (Martin Petersen in particular) worked for many years with the T10 standards committee and storage vendors and implemented Linux kernel support for DIF/DIX, data protection in the Linux kernel, note to those that wonder, yes it's all in mainline Linux and under the GPL. This basically gave us all the features in the Linux kernel to checksum a data block, send it to the storage adapter, which can then validate that block and checksum in firmware before it sends it over the wire to the storage array, which can then do another checksum and to the actual DISK which does a final validation before writing the block to the physical media. So what was missing was the ability for a userspace application (read: Oracle RDBMS) to write a block which then has a checksum and validation all the way down to the disk. application to disk.
Because we have ASMLib we had an entry into the Linux kernel and Martin added support in ASMLib (kernel driver + userspace) for this functionality. Now, this is all based on relatively current Linux kernels, the oracleasm kernel module depends on the main kernel to have support for it so we can make use of it. Thanks to UEK and us having the ability to ship a more modern, current version of the Linux kernel we were able to introduce this feature into ASMLib for Linux from Oracle. This combined with the fact that we build the asm kernel module when we build every single UEK kernel allowed us to continue improving ASMLib and provide it to our customers.
So today, we (Oracle) provide Oracle ASMLib for Oracle Linux and in particular on the Unbreakable Enterprise Kernel. We did the build/testing/delivery of ASMLib for RHEL until RHEL5 but since RHEL6 decided that it was too much effort for us to also maintain all the build and test environments for RHEL and we did not have the ability to use the latest kernel features to introduce the Data Integrity features and we didn't want to end up with multiple versions of asmlib as maintained by us. SuSE SLES still builds and comes with the oracleasm module and they do all the work and RHAT it certainly welcome to do the same. They don't have to rebuild the userspace library, it's really about the kernel module.
And finally to re-iterate a few important things :
hope this helps.
Work on a cluster filesystem at Oracle started many years ago, in the early 2000's when the Oracle Database Cluster development team wrote a cluster filesystem for Windows that was primarily focused on providing an alternative to raw disk devices and help customers with the deployment of Oracle Real Application Cluster (RAC). Oracle RAC is a cluster technology that lets us make a cluster of Oracle Database servers look like one big database. The RDBMS runs on many nodes and they all work on the same data. It's a Shared Disk database design. There are many advantages doing this but I will not go into detail as that is not the purpose of my write up. Suffice it to say that Oracle RAC expects all the database data to be visible in a consistent, coherent way, across all the nodes in the cluster. To do that, there were/are a few options : 1) use raw disk devices that are shared, through SCSI, FC, or iSCSI 2) use a network filesystem (NFS) 3) use a cluster filesystem(CFS) which basically gives you a filesystem that's coherent across all nodes using shared disks. It is sort of (but not quite) combining option 1 and 2 except that you don't do network access to the files, the files are effectively locally visible as if it was a local filesystem.
So OCFS (Oracle Cluster FileSystem) on Windows was born. Since Linux was becoming a very important and popular platform, we decided that we would also make this available on Linux and thus the porting of OCFS/Windows started. The first version of OCFS was really primarily focused on replacing the use of Raw devices with a simple filesystem that lets you create files and provide direct IO to these files to get basically native raw disk performance. The filesystem was not designed to be fully POSIX compliant and it did not have any where near good/decent performance for regular file create/delete/access operations. Cache coherency was easy since it was basically always direct IO down to the disk device and this ensured that any time one issues a write() command it would go directly down to the disk, and not return until the write() was completed. Same for read() any sort of read from a datafile would be a read() operation that went all the way to disk and return. We did not cache any data when it came down to Oracle data files.
So while OCFS worked well for that, since it did not have much of a normal filesystem feel, it was not something that could be submitted to the kernel mail list for inclusion into Linux as another native linux filesystem (setting aside the Windows porting code ...) it did its job well, it was very easy to configure, node membership was simple, locking was disk based (so very slow but it existed), you could create regular files and do regular filesystem operations to a certain extent but anything that was not database data file related was just not very useful in general. Logfiles ok, standard filesystem use, not so much. Up to this point, all the work was done, at Oracle, by Oracle developers.
Once OCFS (1) was out for a while and there was a lot of use in the database RAC world, many customers wanted to do more and were asking for features that you'd expect in a normal native filesystem, a real "general purposes cluster filesystem". So the team sat down and basically started from scratch to implement what's now known as OCFS2 (Oracle Cluster FileSystem release 2). Some basic criteria were :
Needless to say, this was a huge development effort that took many years to complete. A few big milestones happened along the way...
The release model has always been that new feature development happened in the mainline kernel and we then built consistent, well tested, snapshots that had versions, 1.2, 1.4, 1.6, 1.8. But these releases were effectively just snapshots in time that were tested for stability and release quality.
OCFS2 is very easy to use, there's a simple text file that contains the node information (hostname, node number, cluster name) and a file that contains the cluster heartbeat timeouts. It is very small, and very efficient. As Sunil Mushran wrote in the manual :
Because the RHEL kernel did not contain OCFS2 as a kernel module (it is in the source tree but it is not built by the vendor in kernel module form) we stopped adding the extra packages to Oracle Linux and its RHEL compatible kernel and for RHEL. Oracle Linux customers/users obviously get OCFS2 included as part of the Unbreakable Enterprise Kernel, SuSE customers get it by SuSE distributed with SLES and Red Hat can decide to distribute OCFS2 to their customers if they chose to as it's just a matter of compiling the module and making it available.
OCFS2 today, in the mainline kernel is pretty much feature complete in terms of integration with every filesystem feature Linux offers and it is still actively maintained with Joel Becker being the primary maintainer. Since we use OCFS2 as part of Oracle VM, we continue to look at interesting new functionality to add, REFLINK was a good example, and as such we continue to enhance the filesystem where it makes sense. Bugfixes and any sort of code that goes into the mainline Linux kernel that affects filesystems, automatically also modifies OCFS2 so it's in kernel, actively maintained but not a lot of new development happening at this time. We continue to fully support OCFS2 as part of Oracle Linux and the Unbreakable Enterprise Kernel and other vendors make their own decisions on support as it's really a Linux cluster filesystem now more than something that we provide to customers. It really just is part of Linux like EXT3 or BTRFS etc, the OS distribution vendors decide.
Do not confuse OCFS2 with ACFS (ASM cluster Filesystem) also known as Oracle Cloud Filesystem. ACFS is a filesystem that's provided by Oracle on various OS platforms and really integrates into Oracle ASM (Automatic Storage Management). It's a very powerful Cluster Filesystem but it's not distributed as part of the Operating System, it's distributed with the Oracle Database product and installs with and lives inside Oracle ASM. ACFS obviously is fully supported on Linux (Oracle Linux, Red Hat Enterprise Linux) but OCFS2 independently as a native Linux filesystem is also, and continues to also be supported. ACFS is very much tied into the Oracle RDBMS, OCFS2 is just a standard native Linux filesystem with no ties into Oracle products. Customers running the Oracle database and ASM really should consider using ACFS as it also provides storage/clustered volume management. Customers wanting to use a simple, easy to use generic Linux cluster filesystem should consider using OCFS2.
To learn more about OCFS2 in detail, you can find good documentation on http://oss.oracle.com/projects/ocfs2 in the Documentation area, or get the latest mainline kernel from http://kernel.org and read the source.
One final, unrelated note - since I am not always able to publicly answer or respond to comments, I do not want to selectively publish comments from readers. Sometimes I forget to publish comments, sometime I publish them and sometimes I would publish them but if for some reason I cannot publicly comment on them, it becomes a very one-sided stream. So for now I am going to not publish comments from anyone, to be fair to all sides. You are always welcome to email me and I will do my best to respond to technical questions, questions about strategy or direction are sometimes not possible to answer for obvious reasons.
I have quite a few servers at home and I also have 2 hosted servers with a hosted provider. The servers at home I use mostly to play with random Linux related things, or with Oracle VM or just try out various new Oracle products to learn more. I like the technology, it's like a hobby really. To be able to have a good installation experience and use an officially certified Linux distribution and not waste time trying to find the right libraries, I, of course, use Oracle Linux. Now, at least I can get a copy of Oracle Linux for free (even if I was not working for Oracle) and I can/could use that on as many servers at home (or at my company if I worked elsewhere) for testing, development and production. I just go to http://edelivery.oracle.com/linux and download the version(s) I want and off I go.
Now, I also have the right (and not because I am an employee) to take those images and put them on my own server and give them to someone else, I in fact, just recently set up my own mirror on my own hosted server. I don't have to remove oracle-logos, I don't have to rebuild the ISO images, I don't have to recompile anything, I can just put the whole binary distribution on my own server without contract. Perfectly free to do so. Of course the source code of all of this is there, I have a copy of the UEK code at home, just cloned from https://oss.oracle.com/git/?p=linux-2.6-unbreakable.git. And as you can see, the entire changelog, checkins, merges from Linus's tree, complete overview of everything that got changed from kernel to kernel, from patch to patch, errata to errata. No obfuscating, no tar balls and spending time with diff, or go read bug reports to find out what changed (seems silly to me).
Some of my servers are on the external network and I need to be current with security errata, but guess what, no problem, my servers are hooked up to http://public-yum.oracle.com which is open, free, and completely up to date, in a consistent, reliable way with any errata, security or bugfix. So I have nothing to worry about. Also, not because I am an employee. Anyone can. And, with this, I also can, and have, set up my own mirror site that hosts these RPMs. both binary and source rpms. Because I am free to get them and distribute them. I am quite capable of supporting my servers on my own, so I don't need to rely on the support organization so I don't need to have a support subscription :-). So I don't need to pay. Neither would you, at least not with Oracle Linux.
Another cool thing. The hosted servers came (unfortunately) with Centos installed. While Centos works just fine as is, I tend to prefer to be current with my security errata(reliably) and I prefer to just maintain one yum repository instead of 2, I converted them over to Oracle Linux as well (in place) so they happily receive and use the exact same RPMs. Since Oracle Linux is exactly the same from a user/application point of view as RHEL, including files like /etc/redhat-release and no changes from .el. to .centos. I know I have nothing to worry about installing one of the RHEL applications. So, OL everywhere makes my life a lot easier and why not...
Next! Since I run Oracle VM and I have -tons- of VM's on my machines, in some cases on my big WOPR box I have 15-20 VMs running. Well, no problem, OL is free and I don't have to worry about counting the number of VMs, whether it's 1, or 4, or more than 10 ... like some other alternatives started doing...
and finally :) I like to try out new stuff, not 3 year old stuff. So with UEK2 as part of OL6 (and 6.3 in particular) I can play with a 3.0.x based kernel and it just installs and runs perfectly clean with OL6, so quite current stuff in an environment that I know works, no need to toy around with an unsupported pre-alpha upstream distribution with libraries and versions that are not compatible with production software (I have nothing against ubuntu or fedora or opensuse... just not what I can rely on or use for what I need, and I don't need a desktop).
pretty compelling. I say... and again, it doesn't matter that I work for Oracle, if I was working elsewhere, or not at all, all of the above would still apply. Student, teacher, developer, whatever. contrast this with $349 for 2 sockets and oneguest and selfsupport per year to even just get the software bits.
Source RPMs (.srpm) are being uploaded to http://oss.oracle.com/ol6/SRPMS-updates.
OL6.3 contains UEK2 kernel-uek-2.6.39-200.24.1. The source rpm is in the above location but our public GIT repository will be synced up shortly as well at https://oss.oracle.com/git/?p=linux-uek-2.6.39.git;a=summary. Unlike some others, of course, complete source, complete changelog, complete checkin history, both mainline and our own available. No need to go assemble things from a website manually.
Another cool thing coming up is a boot iso for OL6.3 that boots up uek (2.6.39-200.24.1) as install kernel and uses btrfs as the default filesystem for installation. So latest and greatest direct access to btrfs, a modern well-tested, current kernel, freely available. Enjoy.
Since it takes a few days for our ISOs to be on http://edelivery.oracle.com/linux, I mirrored them on my own server :http://mirrors.wimmekes.net/pub/OracleLinux/
Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.
You can follow him on Twitter at @wimcoekaerts


text:
- Home
- Welcome
- Faculty/Staff
- Academic Programs
- Eagle Camera
- Research
- Student Resources
- Contact Us
Welcome to the Department of Biological Sciences at ETSU!
Important Dates and Events
- NEW Advising Procedures: Please Visit the Advising Checklist
- Seminar: Every Wednesday @ 4pm in Brown Hall, Room 304
- Take a look at our newly developed Career Pathways!
Seminars are held on Wednesdays at 4 PM in Brown Hall Room 304
Information concerning the study aboard trip to Galapagos & Ecuador

subset: commoncrawl


text:
Black Christmas
- Studio: Metro-Goldwyn-Mayer (MGM)
- Release Date: Dec 25, 2006
Critic Reviews
- Critic score
- Publication
- By date
The flashbacks are wittily gothic, and the present-day murder scenes have the absurdist, chain-reaction intricacy of the "Final Destination" deaths.
This film is an evocative, effective entry into the holiday blood-spray subgenre in its own right. And if it doesn't make your skin crawl ... you probably ate too much Christmas dinner.
Silly, obvious, clumsy, and just gruesome enough to keep jaded genre fans from angrily throwing popcorn at the screen.
Morgan borrows Christmas-specific nastiness from a wide range of fright flicks, but the result is less than the sum of its parts.
Where the first film was a seminal forerunner of early stalker classics like "Halloween," this version feels as stale as old gingerbread.
Like an ugly tie or a pair of slipper socks, Black Christmas is destined to be forgotten the instant it's unwrapped, gathering dust until the season rolls around again.
There are a couple clever touches here and there, including one sequence in which the end of a candy cane has been carefully licked into a highly lethal weapon, but for the most part the accompanying histrionics feel more regressive than retro.
It exchanges the police subplot that gave the earlier film its steady pace for a lot of pointless backstory about the mother-fixated stalker.
Lazy, perfunctory and free of tension, the new version will satisfy neither the admirers of the original nor anyone looking for a gory respite from seasonal good cheer.
Props to the Weinstein Brothers for having the guts to release a slasher film on Christmas Day. Too bad this one is the cinematic equivalent of tryptophan.
Seems to go out of its way to obliterate all the elements that made the original so special.
Bad film fans will think Christmas has come early, everyone else should ask for the receipt.
With a peephole-riddled set and a flashback-heavy screenplay, Black Christmas smothers terror beneath a blanket of unnecessary information, revealing too much and teasing too little.
It's debatable whether the original 1974 "Black Christmas" is, as its most rabid fans claim, the mother of all slasher movies. But there can be no argument regarding the scant merits of its slapdash, soporifically routine remake, suitable only for the least discriminating of gore hounds.
The remake neither pays perceptive tribute to the original nor updates it in anything but hackneyed form.
The product itself isn't so much afterthought as afterbirth -- a bloody mess to be dumped discreetly.
User Score
Mixed or average reviews- based on 164 Ratings
User score distribution:
Positive: 14 out of 32
Mixed: 3 out of 32
Negative: 15 out of 32
Apr 15, 2011When this movie ended i felt that it missed something on its way, but nonetherless the girls acting were good especially Mary ElizabethWhen this movie ended i felt that it missed something on its way, but nonetherless the girls acting were good especially Mary Elizabeth Winsted, Cristal Lowe and Michelle Trachtenberg… Full Review »
LoganH.Sep 26, 2007I loved the sense of violent tension.
WesleyE.May 7, 2007A surprisingly large amount of fun packaged into what could be viewed as a very mediocre set of decorations on a rotten Christmas ham. One of A surprisingly large amount of fun packaged into what could be viewed as a very mediocre set of decorations on a rotten Christmas ham. One of the better horror remakes of recent times, made all the better by it's lack of morality. Certainly better than the recent The Hills Have Eyes II.… Full Review »


text:
Marvel Assembles an Official Title for Third "Avengers" Movie
Comic Books, Film
Showing results 1-2 of 2
Vampirella #1
Vampirella embarks on a dangerous new mission that immediately goes awry in Nancy A. Collins and Patrick Berkenkotter's "Vampirella" #1.
|June 9th, 2014
X-Treme X-Men #1
"X-Treme X-Men" #1 by Greg Pak and Stephen Segovia has an interesting universe hopping adventure idea that's sadly buried in sloppy writing and art.
|July 27th, 2012


text:
Frequently Asked Questions
2015 NBC BAY AREA TV 35th ANNUAL COMEDY DAY SPOT
Q: Who's performing this year?
A: Check out our performers page.
Q: How do I get there?
A: Go to our directions page.
Q: Where can I park?
A: Check out our parking page.
Q: Where can I stay?
A: Villa Florence...the best boutique hotel on Union Square.
Q: What if it rains?
A: Bring an umbrella because, as they say in showbiz, "The show must go on!" And it will.
Q: Will food be available for sale?
A:You bet! Sam's Chowder Mobile will be serving up Sam's Famous Lobster Roll, Andouille Po' Boy sausage sandwiches, Fried Shrimp or Grilled Prawn Po' Boy sandwiches, fish & chips, fried calamari, and more yummy goodness! Also on board is The Slider Shack, featuring Marin Sun Farms 100% grass fed-beef sliders, Kalua and BBQ pulled pork sliders, Moapfried chicken sliders, Vegetarian black bean sliders, Hurritos (Hawaiian burritos),plus their legendary tasty Tater Tots, and more! Plus ForeverSprung Spring Rolls, featuring Marinated Chicken, Shrimp or Tofu Spring rolls...and coffee. too! And back again this year is the KabobTrolley! They'll be offering gyros & cheesesteaks (dah, again)! And what would Comedy Day be without Ben & Jerry's Ice Cream. Yay! And here's an extra special treat for courageous San Francisco food lovers: Buggin' Out with ChefPV — ChefPV is a full-spectrum chef & food adventurer passionate about food sustainability which means expanding the culinary palates of the people. He is currently promoting Entomophagy as a necessity in a sustainable food system and will be serving Guacamole Chirp Sliders and Chocolate Cricket Protein Shakes. Bug Appétit!
Q: What about beverages?
A: The best beer anywhere from Lagunitas Brewing! They're a a natural fit for Comedy Day because they're known for their iconoclastic interpretations of traditional beer styles, and irreverent descriptive text and stories on their packaging. Brewed in a bizarre old world tradition using only water, hops, yeast, and malted barley. New this year, Common Cider! Wait'll you get your and hands on, and lips around, a bottle of Common Apple Saison, Common Cider Blood Orange Tangerine, Common Hibiscus Saison, or Common Lemon Saison. Brewed in Drytown, CA and using the finest "Apple Hill" region fruit. And our favorite wines (soon to be yours, too) from Pat Paulsen Vineyards. Yes, THAT Pat Paulsen! Pat's son, Monty Paulsen, a graduate of the UC Davis School of Winemaking, has revived the winery and the brand, and is continuing the tradition of producing outstanding, award-winning wines of great value and high quality. And there's be plenty of water and soda...no one goes thirsty at Comedy Day!
Q: Can I buy an official Comedy Day T-shirt at the event?
A: Not this year. However, check out the Comedy Day store for lots of fun stuff including 33rd Annual Comedy Day T-shirts, coffee mugs, teddy bears...even underwear and more! And you like fun T-shirts, be sure to visit NerdKungFu.com for a huge selection of novelty T-shirts—from vintage DC comics to funny political T-shirts.
Q: Can I bring a lawn chair or some other type of seating?
A: We'd prefer you didn't. We're short on space this year. Besides, they ruin the sight lines of those seated around you.
Q: What about the kids? Can we bring 'em?
A: If Comedy Day were a movie...it'd be rated PG-13. We ask our performers to "keep it clean," but occasionally an "off-color" word or topic will pop up. So, please use discretion.
Q: What about my doggie?
A: Sure, why not. Keep 'em on a leash, though, and be sure to clean up after 'em!
Q: Do you provide sign language services?
A: Absolutely...American Sign Language (ASL) thanks to Kylie Kirkpatrick and friends.
Q: Is Comedy Day really FREE?
A: You bet! And it has been since 1981. How do we do it? See below...
Q: Want to contribute to Comedy Day?
A: We could use a few bucks here and there, too! Keep the laughter coming. Comedy Day is a 501(c)(3) nonprofit organization. Your contribution is tax deductible. Our Tax ID number is: 94-2945255. For more information click here. Or check out the Comedy Day store for lots of fun stuff including T-shirts for people (and dogs), mugs, caps, jackets, sweatshirts, hoodies, buttons, tote bags, baby bibs, teddy bears, boxer shorts and briefs...even thongs! And remember, because we're a nonprofit organizations, a portion of your purchase price is tax deductible!
Because Comedy Day is a nonprofit organization, we do not have any paid administrative staff and we rely solely on the efforts of our volunteers. Comedy Day derives income from donors, vendors, sponsors and SF Grants for the Arts. Please patronize our benefactors! Thank you.
Besides putting on the big show each year, Comedy Day also provides an outreach program in which comedy workshops teaching conflict resolution through the use of humor can be conducted in Bay Area public schools. We can also arrange for comedians to perform for charitable organizations, hospitals, nursing homes, assisted-living facilities and hospices throughout the Bay Area. Please contact us!


text:
I tried to do this without success. AFAIK, I should be using wp_get_archives:
First, show the category:
:: Cars
And then, show the post title only of all the posts inside that category, newer ones on top:
:: Cars
-- :: I trashed my Ferrari
-- :: I won a Ferrari!
Of course, I won't have just one category, so this gets harder to do.
If this helps, my permalinks are by /postid/ and then by /posttitle/.
I had seen in the codex the get_archive stuff and it features an example with a dropdown box, so that should be a nice solution when I'll have more than 20 posts in a category.


text:
PHOENIX (March 23, 2014) A 14-year-old girl died and her mother and three siblings were in critical condition after an early-morning apartment fire Sunday in west Phoenix.
Fire department Capt. Jonathan Jacobs says firefighters responded at around at 12:45 a.m. Sunday to a report of a fire in a second floor unit of an apartment building.
He told The Associated Press that firefighters battled heavy smoke as they quickly put on the fire and then found the victims.
Phoenix police Officer James Holmes says the 33-year-old mother and a 10-year-old were hospitalized in critical condition, and a 3-year-old and a 3-month old were in extremely critical but stable condition.
Holmes said the fire may have been intentionally set, and one person has been detained.
Nearly 30 other people who live in the building were displaced.


text:
Hello, Hello. Sign In
Please enter a valid city, state or zip code
14 reviews | 3.9 out of 5
This is a good dock for your money, especially if sound quality is one of your primary concerns. For a unit this size, it produces clean, distortion-free sound at most volume levels, and is more than loud enough for a typically-sized bedroom, office, etc. Bass is full and rich, a concern on units this size, but it does tend to favor bass frequencies closer to midrange over the low end, especially at louder volumes. There are a few compliants I have regarding the day to day use of this dock. The first is that the buttons are arranged in a circular pattern around the dock connector, resulting in some buttons (especially the clock brightness) that are harder to access when your iDevice is docked. The second is the dock's "required" app, which I tried for all of 30 seconds. Thankfully, you don't have to use it, and all of your other audio apps will work great. My iPad would not fit (the rounded top prevented it from fully inserting into the 30 pin connector), but this dock does not claim to support the iPad, so I can't ding it for that. Also, this is an iPhone/iPod dock, so of course your Android device will not work. I would definitely recommend this dock to others.
I ordered this item to replace my bedside alarm clock. My iPhone easily slides onto the top charger, even in the dark. I thought the clock light was way too bright until I figured out it had three brightness settings. It has a cool night light feature around the base that has two brightness settings. It has a small footprint which I like. Forget downloading the Philips alarm app. It's lame, useless and must be running in the foreground. The iPhone's regular alarm works better. I use a very thin iPhone case and its compatable with it and appears it will with most cases. No more setting the clock this item automatically sets itself once you drop a device on it. May buy my wife one to match!
We have had problems with this unit from the first day we bought it. The Fidelio and now Dock Studio software is slow performing and lacks value since you can do everything from the native iPhone apps like Clocks and Music. Currently however the unit will not even recognize the phone so no charging is occurring nor will the time update from the phone to the unit. But worst of all I can't even play music. I have applied for access to the forum to post the bug so if this issue does get fixed then I will update this review. I could not with good faith recommend any of their alarm clock products. There is no faith in reliability.
Has great sound, Very impressed clock synced with my iPhone4s with a case on it!! Like the compactness, I was also impressed the volume control on this unit did also adjust the playing volume directly on iPhone.
I bought this to stream music from my Droid to a speaker. The Philips brand is usually great but this is a let down. To use the blue tooth, you must use the Fidelio app and it doesn't work. It shuts off spontaneously and it drains my battery even when it's not on. I took this product back and tried another Philips product but it also did not have good Bluetooth compatibility with my Droid. It would probably work fine with apple products though.
I've had this speaker dock for 2 years. It worked fine, but recently I updated my iPhone 4 OS and now it is incompatible. I read online that it is a firmware issue. These docks are not cheap, and mine is out of warranty and will not be replaced by Philips. I will not buy another dock or anything similar for the iPhone or iPad. They cost too much and go obsolete too quickly. What a waste of money!
Grandson loved this...
we were looking for a small unit with some sound while using the iPhone for the clock and alarm and needed a charger. this packs a punch and is very simple. charges the iPad, iPhone and iPod, plays through the speaker and been the alarm sounds. with the clock only visible at night one the unit, it is spot on! sound is great! WORKS WITH A CASE, WHICH IS HUGE plus!
Its an out of the box product. Great, clear, crisp sound. (me... Jazz mostly). low light features are very useful. even my kids want to charge their ipods....
giving as a x-mas gift for my grand-daughter, we will see how well it holds up.
Great sound for such a small cabinet. Very happy with the purchase.


text:
Columbus Short will not be returning to Scandal for season 4, Short has confirmed. In a statement to Essence.com, Short says his character, Harrison Wright, who was left at gunpoint during the season 3 finale will not return.
"At this time I must confirm my exit from a show I've called home for 3 years, with what is the most talented ensemble on television today," he wrote.
"I would like to first thank Shonda Rhimes for the opportunity to work with such an amazing cast. Thank you GLADIATORS, who have supported me throughout my entire career and of course to ABC and Shondaland for allowing me to play such a pivotal role in the Scandal series. I have enjoyed every single minute of it."
This marks a sad end to the actor's run on the show, as his character coined the term 'gladiator" in the first episode. Short, who began his career as a choreographer working with acts like Britney Spears, and then acting in memorable roles in the films Stomp The Yard and This Christmas has made all the wrong headlines as of late.
He is currently estranged from his wife, Tanee McCall, who has filed for divorce for the third time and was awarded a restraining order after a fight where she says he threatened to kill her and himself.
Short recently apologized for a now-controversial interview with the Tom Joyner Morning Show the morning after the Scandal season finale.
A-List Stars Who Failed At Marriage
A-List Stars Who Failed At Marriage
1. Usher and Tameka RaymondSource:PR Photos 1 of 14
2. Whitney Houston and Bobby BrownSource:PR Photos 2 of 14
3. Columbus Short and Tanee McCall-ShortSource:PR Photos 3 of 14
4. Ochocinco and Evelyn LozadaSource:PR Photos 4 of 14
5. Tiger Woods and Elin WoodsSource:PR Photos 5 of 14
6. Nicole Murphy and Eddie MurphySource:PR Photos 6 of 14
7. Pilar Sanders and Deion SandersSource:PR Photos 7 of 14
8. Tony Parker and Eva LongoriaSource:PR Photos 8 of 14
9. Kris Humphries and Kim KardashianSource:PR Photos 9 of 14
10. Seal and Heidi KlumSource:PR Photos 10 of 14
11. Taye Diggs and Idina MenzelSource:PR Photos 11 of 14
12. Marc Anthony and Jennifer LopezSource:PR Photos 12 of 14
13. Lamar Odom and Khloe KardashianSource:PR Photos 13 of 14
14. Robin Thicke and Paula PattonSource:PR Photos 14 of 14


text:
From Publishers Weekly
Copyright © Reed Business Information, a division of Reed Elsevier Inc. All rights reserved.
Guinevere is bold and strong, a true heroine in every sense of the word. Guinevere truly gets a rework here. Rather than being the wishy-washy fair-headed maiden of the romantic legend, she is a formidable Queen. (Queen of Happy Endings 20101118)
An enthralling read with fascinating three-dimensional portrayals of Arthurian legendary characters set in a historical accurate Britain. (Laura's Reviews 20101118)
This is quality storytelling that has stood the test of time, and I look forward to seeing the other two volumes in this trilogy back in print. (Historical Novels Reviews 20101123)
Fans of the Arthurian legends will appreciate and love this re-telling. (Debbie's Book Bag 20101129)
It is evident from the beginning that Woolley took her time researching the history of the time period as she goes into extraordinarily vivid description. Her characters come to life. (Rundpinne 20101201)
The world of Arthur and Guinevere was masterfully created. (The Maiden's Court 20101201)
eeing developments through the eyes of Guinevere... gave the story an entirely different feel, rhythm, and emphasis. (Apprentice-Writer 20101206)
Persia Woolley delivers in Child of the Northern Spring... it has garnered itself a spot in my permanent collection. (Read All Over Reviews )
Every little detail was so spot on and it was very impressive. You feel as if you are a part of it. (Books Like Breathing )
Woolley brings light to the to this time period and she paints vivid images of what life was like at the time of the 'Celtic Renaissance' when the many Celtic tribes revolted against marauding Anglo-Saxons. (Books by the Willow Tree )