Archive for the ‘WXS’ Category

WebSphere Extreme Scale & WebSphere Commerce (Part 1)

Monday, May 9th, 2011

Recently we managed to get a customer running WebSphere Commerce V6 on WebSphere 6.1 (had to upgrade from V6), in conjunction with WebSphere Extreme Scale 7.1  It had been a long path and far from as simple as things perhaps made out, just plug it in and away you go not quite like that.  I had written an initial post on setting up WebSphere Extreme Scale and the environment we used, not perhaps realising how far away we were at that point from running this live.

There was still a lot of learning and some quite major changes to make, that you don’t really don’t come across when you use Dynacache within the WebSphere Application Server server for a Commerce environment.

If you are wondering what is Dynacache and you use WebSphere Commerce then it is important that you understand, not running Cache in a Commerce environment is a little like having a fast car and only using your first three gears maybe.  What is possible will always be limited and will not do the environment any good at all, affecting the end users of the site.  You will never really maximise the performance that is possible from your Commerce environment, have a look at this Redbook to get an understanding ‘Mastering Dynacache in WebSphere Commerce‘.  I have also split this blog post into two parts, just so we do not end up with a huge page to read and get bored in, perhaps it could even be three.

The aim of WebSphere Commerce and WebSphere Extreme Scale

This article is not about the details of WXS there are some excellent pieces available such as ‘why WXS‘ for that, especially from Billy Newport. There is also a Elastic Cache redbook which gives some excellent background and also some WC integration options plus part of its team was Jonathan Marshall, who helped us along the WXS path.

For the environment we decided to go for WXS 7.1 and have kept that up to date as well with the latest fixpacks so 7.1.0.2 is the current version.  We then went for the standalone process model rather than using WebSphere to provide the WXS grid, that model is discussed in the redbook.  There are advantages to running on WAS, but it seemed to make sense plus save on licences just using the standalone process mode for the WXS grid on Ubuntu (see previous blog for setup).

The aim of integrating WXS with Commerce was to improve several areas.

  • Capability to hold more cache items as the products available was going to grow substantially on the stores.
  • Reduce the overhead on the application servers when dealing with the cache, such as disk offload and the space the cache takes up within the JVM.
  • Improve startup times for servers when coming on line as the cache would already exist rather than having to build.

For the grid setup we initially have three machines each running a WXS catalogue service process and then having three containers processes on each machines.  This is a starting point and will be further tuned, but it provides the ability to balance in terms of resources and failure.

WebSphere Application Server Configuration

On the application server we had installed WXS onto each machine, and created a catalogue service domain with the three catalogue service processes defined.  It’s worth noting here that the name you give the grid means nothing, you can call it anything you would like.

Also we deployed the enhanced cache monitor rather than the standard version, as this provides a better view of the cache environment when using WXS.

Underneath each of the servers in the environment the following properties need to be added to the Java Virtual Machine custom settings.  They are defining the change in cache environment that will be used and its setup, also the cache is remote.  The first part in the following is the name of the property and after the / is the value.  There may be other properties in there already, depending on your setup.  One setting that was interesting is that you need to tell WXS to turn compression on.

com.ibm.websphere.xs.dynacache.disable_recursive_invalidate  /   true
com.ibm.websphere.xs.dynacache.enable_compression   /    true
com.ibm.websphere.xs.dynacache.ignore_value_in_change_event   /  true
com.ibm.websphere.xs.dynacache.num_initial_containers /  1
com.ibm.websphere.xs.dynacache.topology    /     remote
com.ibm.ws.cache.CacheConfig.cacheProviderName   /  com.ibm.ws.objectgrid.dynacache.CacheProviderImpl
com.ibm.ws.cache.CacheConfig.enableCacheReplication   /  true

One last thing is that when you go into the console even with the fixes applied it will still show the WXS level shown as 7.1.0.0.  So you need to look at the systemout.log on startup to really see what fix level has been applied.

Dynacache Configuration

The dynacache configuration was the point at which, we found that the most serious changes were going to have to be made yet we had assumed it would be minimal.  Problems you will have are not apparent when you run cache within the JVM of the application server, compared to how it will look with a remote grid.  When you use WXS it is the case lots of bad requests or constant chatter for cache content is going to add overhead when the grid is remote.  Even if the grid sits on Gigabit ethernet then ‘remote’ is remote and there will be network overhead.  What we saw initially is that with local dynacache pages would take a second or so to build transfer that to a remote cache page and suddenly it was twelve seconds or more.  Yet the only change was the introduction of WXS, so what could be going on?  What then followed was a lot of time with the support team and here is what we found out.

WebSphere Commerce Data Caches

First we had to implement WebSphere Commerce Data Caches, it is a good article but to me not the most obvious to follow on how you do this.  What it is simply saying is that there are certain types of data you can now hold in cache, that will be generated by commands.  So the cache now goes beyond page fragments and pages, which is what you would typically have setip.  In the case of WXS this is data that could change frequently and have a lot of invalidations, and as we will see lots of invalidation requests are not great.

So rather than holding this data in the grid in this case we want some things to still be held at the application server level, but to do this we have to define additional cache containers.  It is worth noting that making this change will also help your standard Commerce environment with no WXS, especially if you are on V6, where this functionality was added later in the product.  It will be a reason why a lot of sites under V6 with cache will not have this enabled.

In order to do this the first thing that needs creating is a cacheinstance.properties file, you need to put that into the /stores.war/WEB-INF/classes directory in your Commerce deployment.  The file we have have created it looks something like this although it does an area to be tuned in terms of the number of cache entries.  We are creating two containers one general for the DistributedMapCache and the other for ListMemberGroupForUsers, but it can be split how you see fit.  When you look in the Cache Monitor you will see these containers.

cache.instance.0=/services/cache/WCDistributedMapCache
cache.instance.0.cacheSize=6000
cache.instance.0.enableDiskOffload=false
cache.instance.0.flushToDiskOnStop=false
cache.instance.0.useListenerContext=false
cache.instance.0.replicationType=1
cache.instance.0.disableDependencyId=false
cache.instance.0.filterTimeOutInvalidation=true
cache.instance.0.filterLRUInvalidation=true
cache.instance.0.ignoreValueInInvalidationEvent=true
cache.instance.0.disableTemplatesSupport=true
cache.instance.0.useServerClassLoader=true
cache.instance.1=/services/cache/com.ibm.commerce.membergroup.commands.ListMemberGroupsForUserCmdImpl
cache.instance.1.cacheSize=4000
cache.instance.1.enableDiskOffload=false
cache.instance.1.flushToDiskOnStop=false
cache.instance.1.useListenerContext=false
cache.instance.1.replicationType=1
cache.instance.1.disableDependencyId=false
cache.instance.1.filterTimeOutInvalidation=true
cache.instance.1.filterLRUInvalidation=true
cache.instance.1.ignoreValueInInvalidationEvent=true
cache.instance.1.disableTemplatesSupport=true
cache.instance.1.useServerClassLoader=true

With that file deployed then you need to add the content to define the data items into the cachespec.xml.  IBM have provided an example of what data to cache in samples/dynacache/cacheinstances.properties, so take the content defining the commands and copy it into your cachespec.xml, then deploy that update.

Changes to Custom Dynacache Entries

One of the reasons that things had appeared so slow when we enabled WXS was the amount of invalidation requests that were issues, had and the way they were grouped.  This is an important change you would not see locally because there is no overhead.  In the cachespec we had several different stores defined and they were all doing invalidations, on content within them.  It was just something which had built up over time as people had added content into the cachespec.xml file.

Because we listed all the commands that could cause an invalidation to run, what would happen is say we had a basket update the invalidations would run and eighteen different requests would be fired at the WXS environment.  Each one takes time and added together that was taking quite a significant amount of overhead on the request.  So we went back and updated the cachespec so that all the components we were invalidating had the same dependancy-id, and then cut down the invalidations to a single request.

On the dependancy ID that was defined on fragments such as the mini basket, previously each one had its own identifier.  So the dependancy ID was Minicart:storeId:DC_userid which is how the IBM sample file would define user based fragments.  Now all user based fragments in the stores say exactly the same for the ID we will use for invalidation and by doing that they are all removed in a single request on the store rather than multiple requests.

<dependency-id>StoreIDUserIDForInvalidation
<component id=”storeId” type=”parameter”>
<required>true</required>
</component>
<component id=”DC_userId” type=”attribute”>
<required>true</required>
</component>
</dependency-id>

Then after the commands that we would run invalidation such as add to basket, we define a single invalidation.

<invalidation>StoreIDUserIDForInvalidation
<component type=”method” id=”getCommandContext”>
<method>getStoreId</method>
<required>true</required>
</component>
<component type=”method” id=”getCommandContext”>
<method>getUserId</method>
<required>true</required>
</component>
</invalidation>

Once we did this we saw a major change in how the environment performed, again it would even help locally but the measurements would be much smaller periods of time than the remote requests.

Server Restart

Although changes to the cachespec.xml file will be picked up dynamically the other changes will require a server restart.  If your catalogue processes are running in WXS you should see it showing that the grid is active from inside of WAS also in the systemout log file for WAS will show the containers and connections being made.  If the WXS environment is not up and running I have noticed that your WAS server will just stop on startup if the grid is not available.  It does not throw and error it will instead appear to do nothing.

Part II

In the next part I will look at the changes we had to make in the WXS environment from the standard setup plus any other general tips we found useful.

 

Tags: , , , , ,
Posted in eCommerce, WebSphere, WXS | No Comments »

WebSphere Commerce V6 and WebSphere Extreme Scale 7.1

Tuesday, July 27th, 2010

I have written a previous post on migrating to WAS 6.1 under WebSphere Commerce 6, this allows you to then utilise WebSphere Extreme Scale.  But there are then quite a few pieces that you need to pull together to get things going, so hopefully I can give some pointers on what to do.  If it saves you the time and aggravation then its a good thing, if you find anything different then let me know.

First of all I ran up a test environment a standalone 6.1 WAS instance at 6.1.0.31, Commerce at 6.0.0.10 and WebSphere Extreme Scale at 7.1.  Ok right now this is not supported but I could see no reason why it would not work.  We had the WAS/WC environment running on AIX 5.3 and the WXS was remote and installed under Ubunto 6.  We wanted to use the standalone environment for WXS to save us money (no WAS license), and create quite a simple configuration.

Setting up the WAS Side

The first step is to ensure you are patched to the latest levels of WAS 6.1  because of the Commerce migration I had done then the site was running 6.1

Restore the WXS installer onto the environment, and run the installer.

Setting up the Remote Extreme Scale Server

Catalog Server

The catalog

Catalog Container

Extreme Scale Console

The extreme scale console is new in WXS 7.1 and can be used to monitor the environment.

It is started from the <installpath>/ObjectGrid/bin directory using the command startConsoleServer.sh.  one thing I did notice under Ubuntu is that because the install of the software was performed under root, if I used the standard profile it did not now have authority to create the directory structure required under <installpath>/ObjectGrid/console where it wanted to create a number of objects.  This includes a Derby database and a number of files.  I have ended up using sudo to start the console server, but it will be something to address the simplest thing will be to set the right authority.

One the server is started in the browser goto https://<hostname>:7443, use admin and admin as the password and profile.     My first thought is this is quite a nice interface it looks good and the first step is to add the catalog sever.  Enter the hostname the JMX port, which should default to 1099 and the listener port 2809.  No connection will exist until you then create an Extreme Scale Domain.  At first I was unsure should go here because the 7.1 domain documentation only talks about WAS 7.1, but the domain name can be anything.  Once the domain name is added then on the right hand side of the screen add the catalogue server.  The status in the top right hand corner should now go to connected.  If you click around the Commerce site and then go into Monitors > Data Grid Overview you should see DYNACACHE_REMOTE, click that and you will get some details of what is taking place in the cache.

In the environment I am first setting up with a single catalog server and container, then I need to define

Posted in eCommerce, WebSphere, WXS | No Comments »

WebSphere Commerce V6 migration between WAS V6 and V6.1

Tuesday, July 27th, 2010

I am writing a separate post on setting up IBM WebSphere Extreme scale with Commerce, which is going to be really important for you if you have a Commerce site and want to improve your performance.  If you are on Commerce V7 then get Feature Pack 1 installed and you are supported and can try WXS out.  If you are on WebSphere Commerce V6 and that migration all the way to V7 looks ou of the question right now well you can use WXS, but you are going to need to move from WAS 6 to WAS 6.1.  The tasks are in a very loose format  documented in the info centre under Migration, which is a little strange because its talking about migrating from a previous version.  We are not doing that though  because we are on Commerce V6 and we want to change WAS versions.  But it is there just open up the Migration topic and you will find out, then read this alongside it.

Environment & Changes

The migration I was doing was under AIX 5.3 and one import aspect is make sure you set the ulimit value to a high level.  if you don’t and the default we had on our machines was 2000 the migration will fail.  I think its just because of the amount of content that gets moved around, and it just runs out of handlers for it.

So edit the /etc/security/limits file. Add the following lines to the user account that the WebSphere Application Server process runs on:  (for migration this is root).  It can also help the system increasing the default number of files from 2000 to 10000.  if you want you can set these to be unlimited with a -1.

nofiles =  10000
nofiles_hard = 10000
The other aspect is make sure you have a significant amount of space free on the system.  The steps will create various log files and sometimes these can be huge I saw one that was in excess of 10Gb.  We were then getting some very spurious errors about failure to copy an object but nothing that specifically said it failed because of space issues.

Time to allow for Migration

In order to complete the migration you are going to want to allow at least 10 hours.  When I ran the migration on the AIX environment it was a P5 machine with 12Gb memory, so plenty of resources.  It took a long time to complete some of these steps, so don’t expect this to be quick.  I am still working through the ND steps and that may be even longer.

Stand Alone Migration

The first attempt I made at doing the migration was stand alone by that I mean a single WAS server ran my Commerce instance.  In fact had two commerce instances that I wanted to migrate a stage server and a normal server, both existed on the same machine they each ran inside a different WAS profile.

1) Install and then patch WebSphere 6.1 to the latest release

The first thing that you want to do is create yourself a new 6.1 WAS profile.  You can do this either on the Command line or using the Profile Manager tool found under <was61_install_path>/bin

Network Deployment Migration

This was much more complicated and straight away when you read the documentation you will have questions.  The first being Ok it says to migrate my Network deployment manager first, but if you read the instructions will it be any different to just a stand alone server.  Its just not there so instead go into the WAS 6.1 IndoCenter and open this link to migrate the manager.

Ensure that whatever file system you will run the migration on has plenty of space, I would suggest at least 12Gb because of the size of log files that will be created.

1) After installing and patching WAS 6.1 I created a new instance using the instance tool.  I gave it the same node and cell settings as the current manager and also the same ports, as I would expect that the nodes connected to it will want to see the same ports available once migrated.  The documentation under WAS makes it clear that you want to have the same CELL and NODE names, under the Commerce side it does not.

2) If you read the WAS documentation they expect you to use the migration tool that is in the bin directory, they suggest it is the simplest way.  One thing this will do for you is run both the WASPreUpgrade and the WASPostUpgrade commands that are required.  In the Commerce documentation you run the actual commands.

3) Stop your ND Manager under V6

4) Create a backup directory, this will be used by the migration to hold the backed up files and then build from.

5) Run the migration tool, the screens are fairly obvious until you get to the ‘What to do with the applications installed‘ , leave this on the default to migrate and install the applications. On the ‘Directory to Install the Applications‘ again leave the default for the default directory of the target version. Depending on what stage you are at you can either disable or not disable the previous ND Manager version for V6.  If you don’t just be very careful because if you migrate and then start it, and don’t have the V61 version running you will have problems. On the ‘Port value assignments’ take those over from the current version, which will mean you cannot have both ND managers running.

Posted in eCommerce, WebSphere, WXS | No Comments »