a:4:{s:5:"child";a:1:{s:0:"";a:1:{s:3:"rss";a:1:{i:0;a:6:{s:4:"data";s:3:"
";s:7:"attribs";a:1:{s:0:"";a:1:{s:7:"version";s:3:"2.0";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:1:{s:0:"";a:1:{s:7:"channel";a:1:{i:0;a:6:{s:4:"data";s:217:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:1:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:12:"Planet MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:27:"http://www.planetmysql.org/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 23 Nov 2010 00:30:01 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"language";a:1:{i:0;a:5:{s:4:"data";s:2:"en";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:42:"Planet MySQL - http://www.planetmysql.org/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"item";a:50:{i:0;a:6:{s:4:"data";s:83:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:45:"YPDNGG: You Probably Don’t Need Golden Gate";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.bluegecko.net/?p=2283";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:81:"http://www.bluegecko.net/uncategorized/ypdngg-you-probably-dont-need-golden-gate/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:5477:"Before launching into this, I must give due deference to Mogens Nørgaard’s landmark article, You Probably Don’t Need RAC (YPDNR), available here, but originally published Q3 2003 in IOUG Select Journal. Mogens showed that you can be a friend of Oracle without always agreeing with everything they do.
Suddenly, everyone is asking about Golden Gate. In July 2009, Oracle bought Golden Gate Software, just one of several companies that have developed log-based replication mechanisms for Oracle and other databases. This was one of many major acquisitions by Oracle in 2009, including Sun and Relsys. But unlike most of Oracle’s acquisitions, Golden Gate provides very little new functionality not already available in Oracle Streams. Nevertheless, at OpenWorld 2009, Oracle made a shocking announcement. They declared that Golden Gate would be the primary replication channel for Oracle, and that development would cease on Streams and related components.
Usually, Oracle watches these little third-party products for good ideas, then implements them independently (and better) on their own in the Oracle kernel, then watches as the little third-party companies fizzle out. A case in point is direct memory access performance sampling. Precise Software and several other companies in the early 2000s developed low-impact performance sampling and visualization products for Oracle based on sampling the SGA periodically from an external program. In version 10g, Oracle answered them with Active Session History (ASH), which did the same thing but better. Although ASH required customers to purchase the Diagnostic Pack, it still more or less spelled the downfall of competing products.
But in the case of Golden Gate, Oracle already has a log-based replication technology (Streams) built into the kernel, available for a very reasonable price (free with Enterprise Edition). The only major components that Streams lacks compared to Golden Gate is the ability to replicate across database platforms (Oracle to MSSQL, MySQL, etc. and vice versa). Even that capability was clearly around the corner: In 11g, Logical Standby (Data Guard), a technology that uses essentially the same stack of components as Streams, gained cross-platform capabilities.
By 11g, Streams has become a mature and stable product, and is far more scalable and configurable than Golden Gate in many ways. Streams can mine logs on the source or the target, or even a third system. Depending on the load profile, you can use a wide variety of configuration choices, including parallelism at almost any point. Streams also allows customers to choose to enforce transaction order or not.
In contrast, Golden Gate’s parallelism is restricted to the apply side, and in parallel mode, does not have the option of guaranteeing transaction order (it is non-ACID). Golden Gate’s parallel apply splits work up by schema, relying on the assumption that interdependent data at the business process level is confined to a single schema at a time. In other words, if all the tables reside in one schema, then parallel apply doesn’t work, and if they reside in many schemas, the changes in one schema may be applied out of order vis à vis the changes to the other schemas.
Streams is only one of Oracle’s preexisting features that can compete successfully in specific use cases with Golden Gate. Even more ancient and time-tested solutions such as advanced replication and remote materialized views remain supported and highly effective, depending on the requirement.
If you look at many of the use cases where our customers have deployed Golden Gate, I find that the simplest and most scalable engineering solution would have been remote fast-refresh materialized views. Our customers often replicate core look-up data, like exchange rates, inventory levels, and other slowly-changing data between Oracle databases within an enterprise. For this, Golden Gate is completely unjustified, due to cost and complexity compared to remote materialized views. If it were a question of heterogeneous (inter database product) replication, I completely understand. But in the majority of situations where we see Golden Gate in use, it is Oracle to Oracle. Given that, I wonder how it could come to pass that responsible people would recommend and implement a solution for such a requirement involving Golden Gate. Why would Oracle essentially abandon ten years of development and stabilization on a platform like Streams for a less mature, rudimentary product like Golden Gate? Oracle can’t possibly be asking customers to pay additional license fees for a worse version of a product they already own.
So let’s review…
Streams: Mature, complex, requires engineering, highly configurable, scalable, Oracle-only, free.
Golden Gate: Simple, east to deploy, few configuration options, less scalable, expensive, heterogeneous (inter-RDBMS), might break your data.
For me, the corporate direction with regard to Golden Gate is perplexing and smacks of sales-driven (as opposed to requirements and cost-driven) engineering. I can only imagine what it must be like for the team at Oracle that built Log Miner and AQ into an impressive suite of options including Streams.
Related posts:New whitepaper: High availability without breaking the bank
Oracle Support and Certification on AWS
A Cloud over San Francisco for OpenWorld 2010
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 23:01:54 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:9:{i:0;a:5:{s:4:"data";s:27:"Jeremiah Wilton's Oradeblog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:10:"SQL Server";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:13:"Uncategorized";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:3:"fud";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:10:"goldengate";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:10:"replicaion";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:8;a:5:{s:4:"data";s:7:"streams";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:6497:"Before launching into this, I must give due deference to Mogens Nørgaard’s landmark article, You Probably Don’t Need RAC (YPDNR), available here, but originally published Q3 2003 in IOUG Select Journal. Mogens showed that you can be a friend of Oracle without always agreeing with everything they do.
Suddenly, everyone is asking about Golden Gate. In July 2009, Oracle bought Golden Gate Software, just one of several companies that have developed log-based replication mechanisms for Oracle and other databases. This was one of many major acquisitions by Oracle in 2009, including Sun and Relsys. But unlike most of Oracle’s acquisitions, Golden Gate provides very little new functionality not already available in Oracle Streams. Nevertheless, at OpenWorld 2009, Oracle made a shocking announcement. They declared that Golden Gate would be the primary replication channel for Oracle, and that development would cease on Streams and related components.
Usually, Oracle watches these little third-party products for good ideas, then implements them independently (and better) on their own in the Oracle kernel, then watches as the little third-party companies fizzle out. A case in point is direct memory access performance sampling. Precise Software and several other companies in the early 2000s developed low-impact performance sampling and visualization products for Oracle based on sampling the SGA periodically from an external program. In version 10g, Oracle answered them with Active Session History (ASH), which did the same thing but better. Although ASH required customers to purchase the Diagnostic Pack, it still more or less spelled the downfall of competing products.
But in the case of Golden Gate, Oracle already has a log-based replication technology (Streams) built into the kernel, available for a very reasonable price (free with Enterprise Edition). The only major components that Streams lacks compared to Golden Gate is the ability to replicate across database platforms (Oracle to MSSQL, MySQL, etc. and vice versa). Even that capability was clearly around the corner: In 11g, Logical Standby (Data Guard), a technology that uses essentially the same stack of components as Streams, gained cross-platform capabilities.
By 11g, Streams has become a mature and stable product, and is far more scalable and configurable than Golden Gate in many ways. Streams can mine logs on the source or the target, or even a third system. Depending on the load profile, you can use a wide variety of configuration choices, including parallelism at almost any point. Streams also allows customers to choose to enforce transaction order or not.
In contrast, Golden Gate’s parallelism is restricted to the apply side, and in parallel mode, does not have the option of guaranteeing transaction order (it is non-ACID). Golden Gate’s parallel apply splits work up by schema, relying on the assumption that interdependent data at the business process level is confined to a single schema at a time. In other words, if all the tables reside in one schema, then parallel apply doesn’t work, and if they reside in many schemas, the changes in one schema may be applied out of order vis à vis the changes to the other schemas.
Streams is only one of Oracle’s preexisting features that can compete successfully in specific use cases with Golden Gate. Even more ancient and time-tested solutions such as advanced replication and remote materialized views remain supported and highly effective, depending on the requirement.
If you look at many of the use cases where our customers have deployed Golden Gate, I find that the simplest and most scalable engineering solution would have been remote fast-refresh materialized views. Our customers often replicate core look-up data, like exchange rates, inventory levels, and other slowly-changing data between Oracle databases within an enterprise. For this, Golden Gate is completely unjustified, due to cost and complexity compared to remote materialized views. If it were a question of heterogeneous (inter database product) replication, I completely understand. But in the majority of situations where we see Golden Gate in use, it is Oracle to Oracle. Given that, I wonder how it could come to pass that responsible people would recommend and implement a solution for such a requirement involving Golden Gate. Why would Oracle essentially abandon ten years of development and stabilization on a platform like Streams for a less mature, rudimentary product like Golden Gate? Oracle can’t possibly be asking customers to pay additional license fees for a worse version of a product they already own.
So let’s review…
Streams: Mature, complex, requires engineering, highly configurable, scalable, Oracle-only, free.
Golden Gate: Simple, east to deploy, few configuration options, less scalable, expensive, heterogeneous (inter-RDBMS), might break your data.
For me, the corporate direction with regard to Golden Gate is perplexing and smacks of sales-driven (as opposed to requirements and cost-driven) engineering. I can only imagine what it must be like for the team at Oracle that built Log Miner and AQ into an impressive suite of options including Streams.
Related posts:
- New whitepaper: High availability without breaking the bank
- Oracle Support and Certification on AWS
- A Cloud over San Francisco for OpenWorld 2010
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Sarah Novotny";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:1;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:52:"Probably the worst way to deal with a stuck query...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-18337119.post-3720574317044299281";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:98:"http://mysqldatabaseadministration.blogspot.com/2010/11/probably-worst-way-to-deal-with-stuck.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:299:"is to disable a customer's account for more than 24 hours without any warning whatsoever. This happened to one of my accounts and I'm beyond furious at the database and network administrators of HostGator.com. Seriously, guys, I don't know of a more unprofessional way of dealing with a stuck query.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 22:24:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:14:"unprofessional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:9:"hostgator";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:10:"webhosting";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:717:"is to disable a customer's account for more than 24 hours without any warning whatsoever. This happened to one of my accounts and I'm beyond furious at the database and network administrators of HostGator.com. Seriously, guys, I don't know of a more unprofessional way of dealing with a stuck query.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:10:"Frank Mash";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:2;a:6:{s:4:"data";s:43:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:66:"Converting queries with “OR” to “Union” to ulitize indexes";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.arubin.org/blog/?p=69";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:93:"http://www.arubin.org/blog/2010/11/22/converting-queries-with-or-to-union-to-ulitize-indexes/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:4286:"Lets say we have a table storing mail messages and we need to show user’s mailbox: messages sent “from” and “to” the specified user.
Here is our table:
CREATE TABLE `internalmail` (
`i_internalmail` int(10) NOT NULL AUTO_INCREMENT,
`i_senderaddress` int(10) NOT NULL,
`i_recipientaddress` int(10) NOT NULL,
`dt_timestamp` timestamp NULL DEFAULT NULL,
... message body, etc ...
PRIMARY KEY (`i_internalmail`),
) ENGINE=InnoDB DEFAULT CHARSET=latin1
And our query:
select * from internalmail
where (i_senderaddress = 247 or i_recipientaddress = 247 or i_recipientaddress = 0)
and dt_timestamp > '2010-08-01 12:30:47'
order by dt_timestamp desc
In this query we show all messages from and to user_id = 247 plus all messages to system user (user_id=0). We need to show only messages for the last 3 months and show the most recent messages first.
To speed up the query we can try creating indexes:
KEY `i_recipientaddress` (`i_recipientaddress`),
KEY `i_senderaddress` (`i_senderaddress`),
KEY `dt_timestamp` (`dt_timestamp`),
However, as the query uses “OR” those indexes will not work:
mysql> explain select * from internalmail
where (i_senderaddress = 247 or i_recipientaddress = 247 or i_recipientaddress = 0)
and dt_timestamp > '2010-08-01 12:30:47'
order by dt_timestamp desc\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: internalmail
type: ALL
possible_keys: i_recipientaddress,i_senderaddress,dt_timestamp
key: NULL
key_len: NULL
ref: NULL
rows: 4843257
Extra: Using where; Using filesort
1 row in set (0.00 sec)
And original query runs for 3 seconds. To fix this query we can do 2 things:
Rewrite query with UNION instead of OR
Create combined indexes
First, we rewrite query with UNION:
(select * from internalmail where i_senderaddress = 247 and dt_timestamp > ‘2010-08-19 12:30:47′)
union
(select * from internalmail where i_recipientaddress = 247 and dt_timestamp > ‘2010-08-19 12:30:47′)
union
(select * from internalmail where i_recipientaddress = 0 and dt_timestamp > ‘2010-08-19 12:30:47′)
order by dt_timestamp desc;
Second, we create 2 indexes:
mysql> alter table internalmail add key send_dt(i_senderaddress, dt_timestamp);
mysql> alter table internalmail add key recieve_dt(i_recipientaddress, dt_timestamp);
After that, MySQL will be able to fully utilize index for each of the 3 queries in union:
mysql> explain
(select * from internalmail where i_senderaddress = 247 and dt_timestamp > '2010-08-19 12:30:47')
union
(select * from internalmail where i_recipientaddress = 247 and dt_timestamp > '2010-08-19 12:30:47')
union
(select * from internalmail where i_recipientaddress = 0 and dt_timestamp > '2010-08-19 12:30:47')
order by dt_timestamp desc\G
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: internalmail
type: range
possible_keys: i_senderaddress,dt_timestamp,send_dt
key: send_dt
key_len: 9
ref: NULL
rows: 5
Extra: Using where
*************************** 2. row ***************************
id: 2
select_type: UNION
table: internalmail
type: range
possible_keys: i_recipientaddress,dt_timestamp,recieve_dt
key: recieve_dt
key_len: 9
ref: NULL
rows: 11
Extra: Using where
*************************** 3. row ***************************
id: 3
select_type: UNION
table: internalmail
type: range
possible_keys: i_recipientaddress,dt_timestamp,recieve_dt
key: recieve_dt
key_len: 9
ref: NULL
rows: 1
Extra: Using where
*************************** 4. row ***************************
id: NULL
select_type: UNION RESULT
table:
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
Extra: Using filesort
4 rows in set (0.00 sec)
Although this query has to perform a final filesort it is much faster: now it runs in 0 sec compared to 3 seconds originally.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 21:25:04 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:13:"Uncategorized";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4903:"Lets say we have a table storing mail messages and we need to show user’s mailbox: messages sent “from” and “to” the specified user.
Here is our table:
CREATE TABLE `internalmail` (
`i_internalmail` int(10) NOT NULL AUTO_INCREMENT,
`i_senderaddress` int(10) NOT NULL,
`i_recipientaddress` int(10) NOT NULL,
`dt_timestamp` timestamp NULL DEFAULT NULL,
... message body, etc ...
PRIMARY KEY (`i_internalmail`),
) ENGINE=InnoDB DEFAULT CHARSET=latin1
And our query:
select * from internalmail
where (i_senderaddress = 247 or i_recipientaddress = 247 or i_recipientaddress = 0)
and dt_timestamp > '2010-08-01 12:30:47'
order by dt_timestamp desc
In this query we show all messages from and to user_id = 247 plus all messages to system user (user_id=0). We need to show only messages for the last 3 months and show the most recent messages first.
To speed up the query we can try creating indexes:
KEY `i_recipientaddress` (`i_recipientaddress`),
KEY `i_senderaddress` (`i_senderaddress`),
KEY `dt_timestamp` (`dt_timestamp`),
However, as the query uses “OR” those indexes will not work:
mysql> explain select * from internalmail
where (i_senderaddress = 247 or i_recipientaddress = 247 or i_recipientaddress = 0)
and dt_timestamp > '2010-08-01 12:30:47'
order by dt_timestamp desc\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: internalmail
type: ALL
possible_keys: i_recipientaddress,i_senderaddress,dt_timestamp
key: NULL
key_len: NULL
ref: NULL
rows: 4843257
Extra: Using where; Using filesort
1 row in set (0.00 sec)
And original query runs for 3 seconds. To fix this query we can do 2 things:
- Rewrite query with UNION instead of OR
- Create combined indexes
First, we rewrite query with UNION:
(select * from internalmail where i_senderaddress = 247 and dt_timestamp > ‘2010-08-19 12:30:47′)
union
(select * from internalmail where i_recipientaddress = 247 and dt_timestamp > ‘2010-08-19 12:30:47′)
union
(select * from internalmail where i_recipientaddress = 0 and dt_timestamp > ‘2010-08-19 12:30:47′)
order by dt_timestamp desc;
Second, we create 2 indexes:
mysql> alter table internalmail add key send_dt(i_senderaddress, dt_timestamp);
mysql> alter table internalmail add key recieve_dt(i_recipientaddress, dt_timestamp);
After that, MySQL will be able to fully utilize index for each of the 3 queries in union:
mysql> explain
(select * from internalmail where i_senderaddress = 247 and dt_timestamp > '2010-08-19 12:30:47')
union
(select * from internalmail where i_recipientaddress = 247 and dt_timestamp > '2010-08-19 12:30:47')
union
(select * from internalmail where i_recipientaddress = 0 and dt_timestamp > '2010-08-19 12:30:47')
order by dt_timestamp desc\G
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: internalmail
type: range
possible_keys: i_senderaddress,dt_timestamp,send_dt
key: send_dt
key_len: 9
ref: NULL
rows: 5
Extra: Using where
*************************** 2. row ***************************
id: 2
select_type: UNION
table: internalmail
type: range
possible_keys: i_recipientaddress,dt_timestamp,recieve_dt
key: recieve_dt
key_len: 9
ref: NULL
rows: 11
Extra: Using where
*************************** 3. row ***************************
id: 3
select_type: UNION
table: internalmail
type: range
possible_keys: i_recipientaddress,dt_timestamp,recieve_dt
key: recieve_dt
key_len: 9
ref: NULL
rows: 1
Extra: Using where
*************************** 4. row ***************************
id: NULL
select_type: UNION RESULT
table:
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
Extra: Using filesort
4 rows in set (0.00 sec)
Although this query has to perform a final filesort it is much faster: now it runs in 0 sec compared to 3 seconds originally.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Alexander Rubin";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:3;a:6:{s:4:"data";s:73:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:41:"Query caching with MySQL Connector/Python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:35:"http://geert.vanderkelen.org/?p=412";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:38:"http://geert.vanderkelen.org/post/412/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3108:"This blog post shows how to create a cursor class for MySQL Connector/Python which will allow you to cache queries. It will hold the query itself and the result in a global variable.
Note: this is a proof of concept and is only meant as a demonstration on how to extend MySQL Connector/Python.
Why query caching?
You are doing lots of queries that have the same result. It would be expensive to always run the same exact query. MySQL has already a query cache, and there is also memcached. But you like MySQL Connector/Python so much you’d like to do it yourself.
A cursor caching queries and their result
To demonstrate a simple implementation of a query cache, we inherit from an existing class: MySQLCursorBuffered. It will save the executed operation with their results in a ‘global’ variable. We call this cursor MySQLCursorQueryCache.
We take the buffered cursor because we’d like to save the result right away. Below you see we only changed two methods for MySQLCursorBuffered:
.execute(): it will now first check using an md5 checksum whether we executed the query before. If we did, we set the make the cached result active. If not, we simply executed.
._handle_resultset(): called when .execute() did an operation which has a result set. The result we know save in the QUERY_CACHE global dict.
from hashlib import md5
import mysql.connector
QUERY_CACHE = dict()
class MySQLCursorQueryCache(mysql.connector.cursor.MySQLCursorBuffered):
def execute(self, operation, params=None):
self._qmd5 = md5(operation).digest()
if QUERY_CACHE.has_key(self._qmd5):
(self._rows, eof) = QUERY_CACHE[self._qmd5]
self.rowcount = len(self._rows)
self._handle_eof(eof)
self._next_row = 0
else:
super(MySQLCursorQCache, self).execute(operation, params)
def _handle_resultset(self):
(self._rows, eof) = self.db().protocol.get_rows()
self.rowcount = len(self._rows)
self._handle_eof(eof)
self._next_row = 0
QUERY_CACHE[self._qmd5] = (self._rows, eof)
try:
self.db().unread_result = False
except:
pass
self._qmd5 = None
The above code is a proof of concept, there is lots of room for improvement. For example, you need something to invalidate entries in the query cache.
How to use MySQLCursorQueryCache
def main():
cnx = mysql.connector.connect(database='test')
cur = cnx.cursor(cursor_class=MySQLCursorQueryCache)
cur.execute("SELECT NOW()")
print cur.fetchone()
time.sleep(2)
cur.execute("SELECT NOW()")
print cur.fetchone()
cur.close()
cnx.close()
When you would use the default cursor, both executed SQL statements would produce a different result. The above produces the following output:
(datetime.datetime(2010, 11, 22, 21, 20, 4),)
(datetime.datetime(2010, 11, 22, 21, 20, 4),)
When you’d like to have some statements cached, and some not, just create a second cursor cursor_class=MySQLCursorBuffered (see Buffering results with MySQL Connector/Python).";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 20:53:50 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:4:"Work";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:5:"howto";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:8:"myconnpy";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:6:"python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3817:"This blog post shows how to create a cursor class for MySQL Connector/Python which will allow you to cache queries. It will hold the query itself and the result in a global variable.
Note: this is a proof of concept and is only meant as a demonstration on how to extend MySQL Connector/Python.
Why query caching?
You are doing lots of queries that have the same result. It would be expensive to always run the same exact query. MySQL has already a query cache, and there is also memcached. But you like MySQL Connector/Python so much you’d like to do it yourself.
A cursor caching queries and their result
To demonstrate a simple implementation of a query cache, we inherit from an existing class: MySQLCursorBuffered. It will save the executed operation with their results in a ‘global’ variable. We call this cursor MySQLCursorQueryCache.
We take the buffered cursor because we’d like to save the result right away. Below you see we only changed two methods for MySQLCursorBuffered:
- .execute(): it will now first check using an md5 checksum whether we executed the query before. If we did, we set the make the cached result active. If not, we simply executed.
- ._handle_resultset(): called when .execute() did an operation which has a result set. The result we know save in the QUERY_CACHE global dict.
from hashlib import md5
import mysql.connector
QUERY_CACHE = dict()
class MySQLCursorQueryCache(mysql.connector.cursor.MySQLCursorBuffered):
def execute(self, operation, params=None):
self._qmd5 = md5(operation).digest()
if QUERY_CACHE.has_key(self._qmd5):
(self._rows, eof) = QUERY_CACHE[self._qmd5]
self.rowcount = len(self._rows)
self._handle_eof(eof)
self._next_row = 0
else:
super(MySQLCursorQCache, self).execute(operation, params)
def _handle_resultset(self):
(self._rows, eof) = self.db().protocol.get_rows()
self.rowcount = len(self._rows)
self._handle_eof(eof)
self._next_row = 0
QUERY_CACHE[self._qmd5] = (self._rows, eof)
try:
self.db().unread_result = False
except:
pass
self._qmd5 = None
The above code is a proof of concept, there is lots of room for improvement. For example, you need something to invalidate entries in the query cache.
How to use MySQLCursorQueryCache
def main():
cnx = mysql.connector.connect(database='test')
cur = cnx.cursor(cursor_class=MySQLCursorQueryCache)
cur.execute("SELECT NOW()")
print cur.fetchone()
time.sleep(2)
cur.execute("SELECT NOW()")
print cur.fetchone()
cur.close()
cnx.close()
When you would use the default cursor, both executed SQL statements would produce a different result. The above produces the following output:
(datetime.datetime(2010, 11, 22, 21, 20, 4),)
(datetime.datetime(2010, 11, 22, 21, 20, 4),)
When you’d like to have some statements cached, and some not, just create a second cursor cursor_class=MySQLCursorBuffered (see Buffering results with MySQL Connector/Python).
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:17:"Geert Vanderkelen";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:4;a:6:{s:4:"data";s:38:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:33:"2.0.1 Community Release Available";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:70:"http://infinidb.org/infinidb-blog/201-community-release-available.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:70:"http://infinidb.org/infinidb-blog/201-community-release-available.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:412:"We are pleased to announce the availability of the 2.0.1 release of Calpont InfiniDB Community. This is our first maintenance release for 2.0.This release includes a number of bug fixes that you can see at http://bugs.launchpad.net/infinidb. You can download the latest InfiniDB binaries, source code, and updated documentation at: http://infinidb.org/downloads. We welco...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 20:01:57 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:652:"We are pleased to announce the availability of the 2.0.1 release of Calpont InfiniDB Community. This is our first maintenance release for 2.0.
This release includes a number of bug fixes that you can see at http://bugs.launchpad.net/infinidb. You can download the latest InfiniDB binaries, source code, and updated documentation at: http://infinidb.org/downloads. We welco...
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:31:"Calpont & InfiniDB Project Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:5;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:58:"MySQL Editions and Support- what do I get with community?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.bluegecko.net/?p=2274";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:88:"http://www.bluegecko.net/oracle/mysql-editions-and-support-what-do-i-get-with-community/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3266:"The new licensing that was announced by Oracle earlier this month caused some FUD in the community that was addressed last week in an updated graphic comparing support and binary options and blog post from Oracle. However, one of our customers sent me this earlier this week –
We are now re-evaluating whether we want to renew our MySQL Enterprise License. As hard as I tried, I am still confused at what we get from the license we paid other than professional support.
In other words, I don’t know whether we are paying for the binary (for features not found in the Community version) or for the Service? I thought they are two different things. But it is just not clear on their Website.
Do you have some insight into MySQL Enterprise Licensing issue? I mean if it is a matter of buying support, can Blue Gecko provide similar type of Professional Service?
Blue Gecko most often manages MySQL Community Edition and so I spent some time talking to one of the MySQL sales staff members to understand if anything has changed in this edition under this new support structure. I’m happy to say that the Community Edition is alive well, and just as we left it in late October. It still has partitioning, InnoDB and all your favorite engines available in it. There is a new MySQL Cluster Community Edition that has been separated (though still freely available) to parallel the MySQL Cluster Carrier Grade Edition.
What Oracle has changed is to offer MySQL Enterprise Monitor, Enterprise Backup (formerly InnoDB hotbackup), and Cluster Manager software packaged at different levels in addition to Oracle Premier Support.
So, to get to my favorite portion of the question above “Can Blue Gecko provide a similar type of Professional Support?” — The answer is yes and no.
Yes!
Blue Gecko provides operational support and monitoring 24/7 on your databases. Our support encompasses proactive work, maintenance outages, unplanned outages, and performance tuning. We work with existing database staff as either or both front line monitoring response and deep bench technical support. We can also work with a development staff as completely outsourced operations staff. I think by working more closely with a smaller group of clients, we are able to troubleshoot more effectively than an Oracle PS staff member who has never connected to your database.
The “no” portion of the answer is that Blue Gecko does not modify or develop patches to MySQL code. We work with other firms if we find a customer needs a custom patch. Though, as operational DBAs with decades of experience on small and large databases however they’re measured we have found that custom forks and custom patches are seldom the only solution and are most certainly not for the faint of heart.
Blue Gecko’s support is always month-to-month and includes a number of hours each month for the work I outlined above and our staff will be happy to work at odd and inconvenient hours on scheduled and unscheduled work.
You go enjoy your turkey, we’ll be watching your systems.
Related posts:MySQL Enterprise on AWS
Oracle, MySQL, MariaDB, and Drizzle. oh my.
How I got access to My Oracle Support (MOS) for US$2.67
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 20:00:20 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:10:"Remote DBA";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4501:"
The new licensing that was announced by Oracle earlier this month caused some FUD in the community that was addressed last week in an updated graphic comparing support and binary options and blog post from Oracle. However, one of our customers sent me this earlier this week –
We are now re-evaluating whether we want to renew our MySQL Enterprise License. As hard as I tried, I am still confused at what we get from the license we paid other than professional support.
In other words, I don’t know whether we are paying for the binary (for features not found in the Community version) or for the Service? I thought they are two different things. But it is just not clear on their Website.
Do you have some insight into MySQL Enterprise Licensing issue? I mean if it is a matter of buying support, can Blue Gecko provide similar type of Professional Service?
Blue Gecko most often manages MySQL Community Edition and so I spent some time talking to one of the MySQL sales staff members to understand if anything has changed in this edition under this new support structure. I’m happy to say that the Community Edition is alive well, and just as we left it in late October. It still has partitioning, InnoDB and all your favorite engines available in it. There is a new MySQL Cluster Community Edition that has been separated (though still freely available) to parallel the MySQL Cluster Carrier Grade Edition.
What Oracle has changed is to offer MySQL Enterprise Monitor, Enterprise Backup (formerly InnoDB hotbackup), and Cluster Manager software packaged at different levels in addition to Oracle Premier Support.
So, to get to my favorite portion of the question above “Can Blue Gecko provide a similar type of Professional Support?” — The answer is yes and no.
Yes!
Blue Gecko provides operational support and monitoring 24/7 on your databases. Our support encompasses proactive work, maintenance outages, unplanned outages, and performance tuning. We work with existing database staff as either or both front line monitoring response and deep bench technical support. We can also work with a development staff as completely outsourced operations staff. I think by working more closely with a smaller group of clients, we are able to troubleshoot more effectively than an Oracle PS staff member who has never connected to your database.
The “no” portion of the answer is that Blue Gecko does not modify or develop patches to MySQL code. We work with other firms if we find a customer needs a custom patch. Though, as operational DBAs with decades of experience on small and large databases however they’re measured we have found that custom forks and custom patches are seldom the only solution and are most certainly not for the faint of heart.
Blue Gecko’s support is always month-to-month and includes a number of hours each month for the work I outlined above and our staff will be happy to work at odd and inconvenient hours on scheduled and unscheduled work.
You go enjoy your turkey, we’ll be watching your systems.
Related posts:
- MySQL Enterprise on AWS
- Oracle, MySQL, MariaDB, and Drizzle. oh my.
- How I got access to My Oracle Support (MOS) for US$2.67
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Sarah Novotny";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:6;a:6:{s:4:"data";s:73:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:22:"Percona XtraBackup 1.4";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=3803";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:70:"http://www.mysqlperformanceblog.com/2010/11/22/percona-xtrabackup-1-4/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2313:"Percona XtraBackup 1.4 is now available for download.
Version 1.4 fixes problems related to incremental backups. If you do incremental backups, it’s strongly recommended that you upgrade to this release.
Functionality Added or Changed
Incremental backups have changed and now allow the restoration of full backups containing certain rollback transactions that previously caused problems. Please see Preparing the Backups and the --apply-log-only option. (From innobackupex, the --redo-only option should be used). (Yasufumi Kinoshita)
The XtraBackup Test Suite was implemented and is now a standard part of each distribution. (Aleksandr Kuzminsky)
Other New Features:
The --prepare option now reports xtrabackup_binlog_pos_innodb if the information exists. (Yasufumi Kinoshita)
When --prepare is used to restore a partical backup, the data dictionary is now cleaned and contains only tables that exist in the backup. (Yasufumi Kinoshita))
The --table option was extended to accept several regular expression arguments, separated by commas. (Yasufumi Kinoshita)
Other Changes:
Ported to the Percona Server 5.1.47-11 code base. (Yasufumi Kinoshita)
XtraBackup now uses the memory allocators of the host operating system, rather than the built-in InnoDB allocators (see Using Operating System Memory Allocators). (Yasufumi Kinoshita)
Bugs Fixed
Bug #595770 – Binaries are stripped by rpmbuild, so __os_install_post is redefined to change the default behaviour. (Aleksandr Kuzminsky)
Bug #589639 – Fixed a problem of hanging when tablespaces were deleted during the recovery process. (Yasufumi Kinoshita)
Bug #611960 – Fixed a segmentation fault in “xtrabackup”. (Yasufumi Kinoshita)
Miscellaneous important fixes related to incremental backups.
Release Notes for this and previous releases of Percona Xtrabackup can be found in our Wiki.
The latest downloads are available on our website. The latest source code can be found on Launchpad.
Please report any bugs found at Bugs in Percona XtraBackup.
For general questions, use our Percona Discussions Group, and for development questions our Percona Development Group.
For support, commercial, and sponsorship inquiries, contact Percona.
Entry posted by Fred Linhoss |
No comment
Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 19:27:02 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:7:"Backups";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"announce";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:7:"percona";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:7:"release";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:6:"xtradb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5924:"Percona XtraBackup 1.4 is now available for download.
Version 1.4 fixes problems related to incremental backups. If you do incremental backups, it’s strongly recommended that you upgrade to this release.
Functionality Added or Changed
- Incremental backups have changed and now allow the restoration of full backups containing certain rollback transactions that previously caused problems. Please see Preparing the Backups and the --apply-log-only option. (From innobackupex, the --redo-only option should be used). (Yasufumi Kinoshita)
- The XtraBackup Test Suite was implemented and is now a standard part of each distribution. (Aleksandr Kuzminsky)
- Other New Features:
- The --prepare option now reports xtrabackup_binlog_pos_innodb if the information exists. (Yasufumi Kinoshita)
- When --prepare is used to restore a partical backup, the data dictionary is now cleaned and contains only tables that exist in the backup. (Yasufumi Kinoshita))
- The --table option was extended to accept several regular expression arguments, separated by commas. (Yasufumi Kinoshita)
- Other Changes:
- Ported to the Percona Server 5.1.47-11 code base. (Yasufumi Kinoshita)
- XtraBackup now uses the memory allocators of the host operating system, rather than the built-in InnoDB allocators (see Using Operating System Memory Allocators). (Yasufumi Kinoshita)
Bugs Fixed
- Bug #595770 – Binaries are stripped by rpmbuild, so __os_install_post is redefined to change the default behaviour. (Aleksandr Kuzminsky)
- Bug #589639 – Fixed a problem of hanging when tablespaces were deleted during the recovery process. (Yasufumi Kinoshita)
- Bug #611960 – Fixed a segmentation fault in “xtrabackup”. (Yasufumi Kinoshita)
- Miscellaneous important fixes related to incremental backups.
Release Notes for this and previous releases of Percona Xtrabackup can be found in our Wiki.
The latest downloads are available on our website. The latest source code can be found on Launchpad.
Please report any bugs found at Bugs in Percona XtraBackup.
For general questions, use our Percona Discussions Group, and for development questions our Percona Development Group.
For support, commercial, and sponsorship inquiries, contact Percona.
Entry posted by Fred Linhoss |
No comment
Add to: 
| 
| 
| 
| 
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:7;a:6:{s:4:"data";s:38:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:37:"Expect a SkySQL price list early 2011";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:69:"tag:blogger.com,1999:blog-9127647805974709324.post-401311074702518784";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:72:"http://blogs.skysql.com/2010/11/expect-skysql-price-list-early-2011.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:394:"We have been asked about when you should expect the SkySQL price list to go live: This will be in January 2011.While we're on the topic of pricing: We do plan to maintain an aggressive pricing policy, as witnessed by our limited-time Gold offer to purchase a SkySQL™ Enterprise Platinum support subscription at the historical MySQL® Enterprise Gold subscription price of just USD 2999.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 15:20:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:857:"We have been asked about when you should expect the SkySQL price list to go live: This will be in January 2011.
While we're on the topic of pricing: We do plan to maintain an aggressive pricing policy, as witnessed by our limited-time Gold offer to purchase a SkySQL™ Enterprise Platinum support subscription at the historical MySQL® Enterprise Gold subscription price of just USD 2999.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Kathleen ONeil";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:8;a:6:{s:4:"data";s:73:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:33:"Unexpected mysqld crashing in 5.5";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=3366";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:76:"http://ronaldbradford.com/blog/unexpected-mysqld-crashing-in-5-5-2010-11-22/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:813:"An update of MySQL from 5.0 to 5.5 on CentOS 5.5 64bit has not resulted in a good experience. The mysqld process would then crash every few minutes with the following message.
101120 8:29:27 InnoDB: Operating system error number 22 in a file operation.
InnoDB: Error number 22 means ‘Invalid argument’.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: File name /tmpfs/#sql6cf3_5c_0.ibd
InnoDB: File operation call: ‘aio write’.
InnoDB: Cannot continue operation.
The work around was to change the tmpdir=/tmpfs (which was a 16G tmpfs filesystem) to a physical disk.
The referenced URL didn’t provide any more information of help. Unlike Bug #26662 O_DIRECT is not specified as the flush method.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 13:41:25 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:3:"5.5";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:5:"crash";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:6:"tmpdir";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:5:"tmpfs";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1111:"An update of MySQL from 5.0 to 5.5 on CentOS 5.5 64bit has not resulted in a good experience. The mysqld process would then crash every few minutes with the following message.
101120 8:29:27 InnoDB: Operating system error number 22 in a file operation.
InnoDB: Error number 22 means ‘Invalid argument’.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/operating-system-error-codes.html
InnoDB: File name /tmpfs/#sql6cf3_5c_0.ibd
InnoDB: File operation call: ‘aio write’.
InnoDB: Cannot continue operation.
The work around was to change the tmpdir=/tmpfs (which was a 16G tmpfs filesystem) to a physical disk.
The referenced URL didn’t provide any more information of help. Unlike Bug #26662 O_DIRECT is not specified as the flush method.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:9;a:6:{s:4:"data";s:58:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:26:"Damm you Peformance Schema";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=3360";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:69:"http://ronaldbradford.com/blog/damm-you-peformance-schema-2010-11-22/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2106:"One significant new feature of MySQL 5.5 is the Performance Schema. I recently performed an upgrade from 5.0 to 5.5, however my check of differences in the MySQL variables via mysqladmin variables failed because we now have a new record long variable name “performance_schema_events_waits_history_long_size”.
The full list of new performance_schema variables are:
> | performance_schema_events_waits_history_long_size | 10000 |
> | performance_schema_events_waits_history_size | 10 |
> | performance_schema_max_cond_classes | 80 |
> | performance_schema_max_cond_instances | 1000 |
> | performance_schema_max_file_classes | 50 |
> | performance_schema_max_file_handles | 32768 |
> | performance_schema_max_file_instances | 10000 |
> | performance_schema_max_mutex_classes | 200 |
> | performance_schema_max_mutex_instances | 1000000 |
> | performance_schema_max_rwlock_classes | 30
Not really a big problem, and not a complaint, just no longer a simple diff.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 13:30:58 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:18:"performance schema";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2428:"One significant new feature of MySQL 5.5 is the Performance Schema. I recently performed an upgrade from 5.0 to 5.5, however my check of differences in the MySQL variables via mysqladmin variables failed because we now have a new record long variable name “performance_schema_events_waits_history_long_size”.
The full list of new performance_schema variables are:
> | performance_schema_events_waits_history_long_size | 10000 |
> | performance_schema_events_waits_history_size | 10 |
> | performance_schema_max_cond_classes | 80 |
> | performance_schema_max_cond_instances | 1000 |
> | performance_schema_max_file_classes | 50 |
> | performance_schema_max_file_handles | 32768 |
> | performance_schema_max_file_instances | 10000 |
> | performance_schema_max_mutex_classes | 200 |
> | performance_schema_max_mutex_instances | 1000000 |
> | performance_schema_max_rwlock_classes | 30
Not really a big problem, and not a complaint, just no longer a simple diff.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:10;a:6:{s:4:"data";s:88:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:22:"Recently in MariaDB #2";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:31:"http://askmonty.org/blog/?p=178";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:47:"http://askmonty.org/blog/recently-in-mariadb-2/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1781:"There has been a lot happening in the MariaDB community recently, and there has been growth. Here are some of the highlights. Thank you to all our current contributors, and to others that want to contribute, shoot community[at]askmonty[dot]org an e-mail.
MariaDB 5.2.3 binaries for Solaris and Debian Sparc
Our Sparc community contributor, Mark, has continued to make popular binaries for Solaris 10 and Debian Sparc. He’s kept up to speed with MariaDB 5.2.3, so please visit him and download the binaries.
MariaDB 5.2.3 on the openSUSE Build Service
Community contributor Michal Hrušecký has packaged MariaDB for openSUSE and its available via the openSUSE build service. It is ready up to version 5.2.3, and is still in the unstable repository (it will progress further based on more testing). A benefit of the openSUSE build service is that it also churns out RPMs for CentOS, Fedora, Mandriva, Red Hat Enterprise Linux, openSUSE and SUSE Linux Enterprise.
MariaDB Virtual Machines
Again from Mark, he’s been working on creating virtual machines for people to test MariaDB with. He’s looking for feedback on which virtualization platform people are looking for VMs for. Currently there is a VM for CentOS 5.5 and Ubuntu 10.04 LTS. These are all 64-bit virtual machines. You can make use of them using VirtualBox.
Mirrors
Looking back at the releases page, we have made eleven 5.1 releases and four 5.2 releases. When we made our first 5.1 release we had one mirror (this was in April 2009). When we made our first 5.2 release, we had four mirrors (this was in April 2010). Today we are up to seven mirrors, and from recent requests I expect this to grow. If you want to mirror MariaDB, check out the Mirroring MariaDB guide on the AskMonty Knowledgebase.
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 05:01:58 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:10:{i:0;a:5:{s:4:"data";s:9:"Community";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Debian";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:7:"MariaDB";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"mirrors";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:8:"openSUSE";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:22:"openSUSE Build Service";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:12:"Planet MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:10:"Solaris 10";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:8;a:5:{s:4:"data";s:5:"Sparc";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:9;a:5:{s:4:"data";s:16:"Virtual Machines";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2881:"There has been a lot happening in the MariaDB community recently, and there has been growth. Here are some of the highlights. Thank you to all our current contributors, and to others that want to contribute, shoot community[at]askmonty[dot]org an e-mail.
MariaDB 5.2.3 binaries for Solaris and Debian Sparc
Our Sparc community contributor, Mark, has continued to make popular binaries for Solaris 10 and Debian Sparc. He’s kept up to speed with MariaDB 5.2.3, so please visit him and download the binaries.
MariaDB 5.2.3 on the openSUSE Build Service
Community contributor Michal Hrušecký has packaged MariaDB for openSUSE and its available via the openSUSE build service. It is ready up to version 5.2.3, and is still in the unstable repository (it will progress further based on more testing). A benefit of the openSUSE build service is that it also churns out RPMs for CentOS, Fedora, Mandriva, Red Hat Enterprise Linux, openSUSE and SUSE Linux Enterprise.
MariaDB Virtual Machines
Again from Mark, he’s been working on creating virtual machines for people to test MariaDB with. He’s looking for feedback on which virtualization platform people are looking for VMs for. Currently there is a VM for CentOS 5.5 and Ubuntu 10.04 LTS. These are all 64-bit virtual machines. You can make use of them using VirtualBox.
Mirrors
Looking back at the releases page, we have made eleven 5.1 releases and four 5.2 releases. When we made our first 5.1 release we had one mirror (this was in April 2009). When we made our first 5.2 release, we had four mirrors (this was in April 2010). Today we are up to seven mirrors, and from recent requests I expect this to grow. If you want to mirror MariaDB, check out the Mirroring MariaDB guide on the AskMonty Knowledgebase.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:24:"Monty Program Group Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:11;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:23:"MariaDB 5.1.51 released";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:31:"http://askmonty.org/blog/?p=176";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:49:"http://askmonty.org/blog/mariadb-5-1-51-released/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:721:"Dear MariaDB users,
The development team have been busy bees, and have freshly released MariaDB 5.1.51. MariaDB is a branch of the MySQL database which includes all major open source storage engines, myriad bug fixes and many community patches.
This release of MariaDB 5.1.51 includes MySQL up to version 5.1.51. It also includes XtraDB up to version 5.1.51-12. For more information, do not hesitate to read the release notes, and for more detailed changes, do hop on over to the changelog.
MariaDB is available in source and binary form in a variety of platforms, including Microsoft Windows, Debian, Ubuntu, Red Hat Enterprise Linux 5, CentOS 5, and Solaris x86. These are all available from the downloads page.
Enjoy!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 22 Nov 2010 04:23:17 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:12:"Announcement";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:7:"MariaDB";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:14:"MariaDB 5.1.51";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1247:"Dear MariaDB users,
The development team have been busy bees, and have freshly released MariaDB 5.1.51. MariaDB is a branch of the MySQL database which includes all major open source storage engines, myriad bug fixes and many community patches.
This release of MariaDB 5.1.51 includes MySQL up to version 5.1.51. It also includes XtraDB up to version 5.1.51-12. For more information, do not hesitate to read the release notes, and for more detailed changes, do hop on over to the changelog.
MariaDB is available in source and binary form in a variety of platforms, including Microsoft Windows, Debian, Ubuntu, Red Hat Enterprise Linux 5, CentOS 5, and Solaris x86. These are all available from the downloads page.
Enjoy!
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:24:"Monty Program Group Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:12;a:6:{s:4:"data";s:73:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:45:"Buffering results with MySQL Connector/Python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:35:"http://geert.vanderkelen.org/?p=388";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:38:"http://geert.vanderkelen.org/post/388/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2281:"MySQL Connector/Python doesn’t buffer results by default. This means you have to fetch the rows when you issued a SELECT. This post describes how you can change this behavior.
Why buffering result sets?
Buffering or storing the result set on the client side is handy when you, for example, would like to use multiple cursors per connection and you’de like to traverse each one interleaved.
Keep in mind that with bigger result sets, the client side will use more memory. You just need to find out for yourself what’s best. When you know result sets are mostly small, you might opt to buffer.
MySQLdb by default buffers results and you need to use a different cursor to disable it. oursql does not buffer by default. This is good to know when you start using a different MySQL DB API for Python.
Use case: 1 connection, fetch from 2 cursors
You want to fetch data from two tables and process the data using one connection. If you do it without buffering, you would run into the following:
>>> cnx = mysql.connector.connect(database='test')
>>> cur1 = cnx.cursor()
>>> cur2 = cnx.cursor()
>>> cur1.execute("SELECT c1 FROM t1")
-1
>>> cur2.execute("SELECT c1 FROM t1")
..
mysql.connector.errors.InternalError: Unread result found.
MySQL Connector/Python offers two ways to turn buffering on or off. Either per connection or per cursor using the buffered argument set to True
Set buffering per connection
If you want all the cursors for a particular connection to be buffered, you can turn it on when connecting to MySQL setting the buffered-argument to True.
>>> import mysql.connector
>>> cnx = mysql.connector.connect(database='test',buffered=True)
>>> cur1 = cnx.cursor()
>>> cur1.__class__
>>> cur2 = cnx.cursor()
>>> cur1.execute("SELECT c1 FROM t1")
3
>>> cur2.execute("SELECT c1 FROM t1")
3
>>> cur1.fetchone()
(u'Geert',)
>>> cur2.fetchone()
Set buffering per cursor
You can request a buffering cursor buffering from the connection object. Do this use the buffered-argument set to True:
>>> import mysql.connector
>>> cnx = mysql.connector.connect(database='test')
>>> cur1 = cnx.cursor(buffered=True)
>>> cur1.__class__
<class 'mysql.connector.cursor.MySQLCursorBuffered'>
Note: the above should work with MySQL Connector/Python v0.2 or greater";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 21 Nov 2010 22:54:27 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:4:"Work";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:5:"howto";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:8:"myconnpy";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:6:"python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2938:"MySQL Connector/Python doesn’t buffer results by default. This means you have to fetch the rows when you issued a SELECT. This post describes how you can change this behavior.
Why buffering result sets?
Buffering or storing the result set on the client side is handy when you, for example, would like to use multiple cursors per connection and you’de like to traverse each one interleaved.
Keep in mind that with bigger result sets, the client side will use more memory. You just need to find out for yourself what’s best. When you know result sets are mostly small, you might opt to buffer.
MySQLdb by default buffers results and you need to use a different cursor to disable it. oursql does not buffer by default. This is good to know when you start using a different MySQL DB API for Python.
Use case: 1 connection, fetch from 2 cursors
You want to fetch data from two tables and process the data using one connection. If you do it without buffering, you would run into the following:
>>> cnx = mysql.connector.connect(database='test')
>>> cur1 = cnx.cursor()
>>> cur2 = cnx.cursor()
>>> cur1.execute("SELECT c1 FROM t1")
-1
>>> cur2.execute("SELECT c1 FROM t1")
..
mysql.connector.errors.InternalError: Unread result found.
MySQL Connector/Python offers two ways to turn buffering on or off. Either per connection or per cursor using the buffered argument set to True
Set buffering per connection
If you want all the cursors for a particular connection to be buffered, you can turn it on when connecting to MySQL setting the buffered-argument to True.
>>> import mysql.connector
>>> cnx = mysql.connector.connect(database='test',buffered=True)
>>> cur1 = cnx.cursor()
>>> cur1.__class__
>>> cur2 = cnx.cursor()
>>> cur1.execute("SELECT c1 FROM t1")
3
>>> cur2.execute("SELECT c1 FROM t1")
3
>>> cur1.fetchone()
(u'Geert',)
>>> cur2.fetchone()
Set buffering per cursor
You can request a buffering cursor buffering from the connection object. Do this use the buffered-argument set to True:
>>> import mysql.connector
>>> cnx = mysql.connector.connect(database='test')
>>> cur1 = cnx.cursor(buffered=True)
>>> cur1.__class__
<class 'mysql.connector.cursor.MySQLCursorBuffered'>
Note: the above should work with MySQL Connector/Python v0.2 or greater
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:17:"Geert Vanderkelen";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:13;a:6:{s:4:"data";s:58:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:33:"How MySQL Workbench breaks itself";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:58:"tag:blogger.com,1999:blog-16959946.post-977563335068143061";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:78:"http://datacharmer.blogspot.com/2010/11/how-mysql-workbench-breaks-itself.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1110:"Once upon a time, there was a policy in MySQL not to add new features after the beta stage.To my surprise, MySQL Workbench 5.2.30 introduces a new feature, the query formatter. I gave it a try. The results are not extremely encouraging. Granted, it's a plugin and not a feature in the core application, but nonetheless one would expect something more stable in a GA release, especially since the plugin features are displayed in the main menu, and unless you have read the announcement, you couldn't easily tell the core from the plugins.This is what I have got in just a few minutes: Bug #58356: beautify function fails on CREATE TABLE Bug #58357: beutify function erases statement on CREATE INDEX Bug #58358: query formatter fails on selected query Bug #58359: query formatter indentation fails on partially selected query Bug #58360 query formatter converts non-keywords to uppercase Bug #58361 Query formatter mangles query when CASE operator is usedMySQL Workbench is a great product. I would like it to be more solid. New features, even as a plugin, should be more carefully released that this one.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 21 Nov 2010 16:32:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:4:"bugs";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:7:"testing";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:9:"workbench";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1876:"Once upon a time, there was a policy in MySQL not to add new features after the beta stage.
To my surprise, MySQL Workbench 5.2.30 introduces a new feature, the query formatter. I gave it a try. The results are not extremely encouraging. Granted, it's a plugin and not a feature in the core application, but nonetheless one would expect something more stable in a GA release, especially since the plugin features are displayed in the main menu, and unless you have read the announcement, you couldn't easily tell the core from the plugins.
This is what I have got in just a few minutes:
Bug #58356: beautify function fails on CREATE TABLE
Bug #58357: beutify function erases statement on CREATE INDEX
Bug #58358: query formatter fails on selected query
Bug #58359: query formatter indentation fails on partially selected query
Bug #58360 query formatter converts non-keywords to uppercase
Bug #58361 Query formatter mangles query when CASE operator is used
MySQL Workbench is a great product. I would like it to be more solid. New features, even as a plugin, should be more carefully released that this one.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Giuseppe Maxia";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:14;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:26:"Clearing the global status";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.linuxjedi.co.uk/?p=63";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.linuxjedi.co.uk/?p=63";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:616:"If you want to clear local session status variables this is easy, simply execute FLUSH STATUS. Unfortunately doing the same for global status variables is not as easy.
That is until now. In the Drizzle7 beta release 2010-11-08 is a new command: FLUSH GLOBAL STATUS
I created this command to fulfil a Rackspace feature request but it appears this has also been a MySQL feature request in the past. The implementation would be very different in MySQL since the status variables and sessions work differently, but it would certainly be good to see in a future release, and it appears patches already exist for it.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 21 Nov 2010 14:24:51 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:7:"Drizzle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:9:"Rackspace";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1084:"If you want to clear local session status variables this is easy, simply execute FLUSH STATUS. Unfortunately doing the same for global status variables is not as easy.
That is until now. In the Drizzle7 beta release 2010-11-08 is a new command: FLUSH GLOBAL STATUS
I created this command to fulfil a Rackspace feature request but it appears this has also been a MySQL feature request in the past. The implementation would be very different in MySQL since the status variables and sessions work differently, but it would certainly be good to see in a future release, and it appears patches already exist for it.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:16:"Andrew Hutchings";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:15;a:6:{s:4:"data";s:43:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:21:"HeidiSQL 6.0 released";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:48:"http://www.heidisql.com/rss.php?c=1,7&p=6948";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:46:"http://www.heidisql.com/forum.php?t=6948#p6948";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1043:"This is a new release of HeidiSQL, the result of 331 changes done after the previous 5.1 release. HeidiSQL 6.0 introduces many new features of which I'll list the most important ones here:- Tabs for multiple batch results or multiple procedure results in a query tab.- Multiple connections in one application instance- Editable query results- Rewritten user manager, using GRANT and REVOKE syntax now- Rewritten "Copy table" dialog and logic, supporting WHERE filters now- Rewritten "Load text file" dialog and logic, supports client side parsing for situations where LOAD DATA is not allowed- Rewritten SQL help dialog- Query profiling in query tabs- Introduce "Run current query" action (Shift+Ctrl+F9)- Enhance detection of Unicode text files- Server details hint when mouse over version status panel- Internal image previewer for BLOBs- LaTeX and Wiki markup output for grids- ZEROFILL support in table editor- Session manager: Prompt for username and password on demand- Add support for TIME values above 24 hours- Tons of bugfixesGrab it";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sat, 20 Nov 2010 18:35:16 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:12:"Anouncements";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1421:"This is a new release of HeidiSQL, the result of 331 changes done after the previous 5.1 release. HeidiSQL 6.0 introduces many new features of which I'll list the most important ones here:
- Tabs for multiple batch results or multiple procedure results in a query tab.
- Multiple connections in one application instance
- Editable query results
- Rewritten user manager, using GRANT and REVOKE syntax now
- Rewritten "Copy table" dialog and logic, supporting WHERE filters now
- Rewritten "Load text file" dialog and logic, supports client side parsing for situations where LOAD DATA is not allowed
- Rewritten SQL help dialog
- Query profiling in query tabs
- Introduce "Run current query" action (Shift+Ctrl+F9)
- Enhance detection of Unicode text files
- Server details hint when mouse over version status panel
- Internal image previewer for BLOBs
- LaTeX and Wiki markup output for grids
- ZEROFILL support in table editor
- Session manager: Prompt for username and password on demand
- Add support for TIME values above 24 hours
- Tons of bugfixes
Grab it
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Ansgar Becker";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:16;a:6:{s:4:"data";s:58:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:35:"MySQL Workbench 5.2.30 GA Available";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:26:"http://wb.mysql.com/?p=833";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:26:"http://wb.mysql.com/?p=833";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2145:"We’re proud to announce the next release of MySQL Workbench, version 5.2.30. This is a maintenance release focusing on general product improvement and usability. In addition to that, we have added a small set of Text-Formatting-Plugins to the SQL Editor to help format/structure your SQL Queries.
As always, we want to thank everyone for the great feedback we have received. This helps us to continuously improve the functionality and stability of MySQL Workbench – we appreciate all your ideas for improving MySQL Workbench. Please keep sending us your ideas!
About the Query-Formatter
In the Edit->Format menu of Workbench’s SQL Editor you will see a set of functions that assist in reformatting your SQL Queries, for example “Beautify Query”, “UPCASE Keywords”, etc.
MySQL Workbench 5.2 GA
Data Modeling
Query (replaces the old MySQL Query Browser)
Administration (replaces the old MySQL Administrator)
Please get your copy from our Download site. Sources and binary packages are available for several platforms, including Windows, Mac OS X and Linux.
http://dev.mysql.com/downloads/workbench/
To get started quickly, please take a look at this short tutorial.
MySQL Workbench 5.2 RC Tutorial
http://wb.mysql.com/?p=406
Workbench Documentation can be found here.
http://dev.mysql.com/doc/workbench/en/index.html
In addition to the new Query/SQL Development and Administration modules, version 5.2 features improved stability and performance – especially in Windows, where OpenGL support has been enhanced and the UI was optimized to offer better responsiveness.
This release also includes improvements to the scripting capabilities of the SQL Editor. You can read more about it in
http://wb.mysql.com/workbench/doc/
For a detailed list of resolved issues, see the change log.
http://dev.mysql.com/doc/workbench/en/wb-change-history.html
If you need any additional info or help please get in touch with us.
Post in our forums, leave comments on our blog pages or if you want to talk to us directly you can visit us on our IRC channel #workbench on irc.freenode.net.
- The MySQL Workbench Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sat, 20 Nov 2010 08:57:26 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:13:"Announcements";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"Linux";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:3:"Mac";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"Windows";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2823:"We’re proud to announce the next release of MySQL Workbench, version 5.2.30. This is a maintenance release focusing on general product improvement and usability. In addition to that, we have added a small set of Text-Formatting-Plugins to the SQL Editor to help format/structure your SQL Queries.
As always, we want to thank everyone for the great feedback we have received. This helps us to continuously improve the functionality and stability of MySQL Workbench – we appreciate all your ideas for improving MySQL Workbench. Please keep sending us your ideas!
About the Query-Formatter
In the Edit->Format menu of Workbench’s SQL Editor you will see a set of functions that assist in reformatting your SQL Queries, for example “Beautify Query”, “UPCASE Keywords”, etc.
MySQL Workbench 5.2 GA
- Data Modeling
- Query (replaces the old MySQL Query Browser)
- Administration (replaces the old MySQL Administrator)
Please get your copy from our Download site. Sources and binary packages are available for several platforms, including Windows, Mac OS X and Linux.
http://dev.mysql.com/downloads/workbench/
To get started quickly, please take a look at this short tutorial.
MySQL Workbench 5.2 RC Tutorial
http://wb.mysql.com/?p=406
Workbench Documentation can be found here.
http://dev.mysql.com/doc/workbench/en/index.html
In addition to the new Query/SQL Development and Administration modules, version 5.2 features improved stability and performance – especially in Windows, where OpenGL support has been enhanced and the UI was optimized to offer better responsiveness.
This release also includes improvements to the scripting capabilities of the SQL Editor. You can read more about it in
http://wb.mysql.com/workbench/doc/
For a detailed list of resolved issues, see the change log.
http://dev.mysql.com/doc/workbench/en/wb-change-history.html
If you need any additional info or help please get in touch with us.
Post in our forums, leave comments on our blog pages or if you want to talk to us directly you can visit us on our IRC channel #workbench on irc.freenode.net.
- The MySQL Workbench Team
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:18:"The Workbench Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:17;a:6:{s:4:"data";s:43:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:15:"Chris on SkySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:35:"http://www.chriscalender.com/?p=151";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:35:"http://www.chriscalender.com/?p=151";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:533:"As some of you already know, I too have recently left the MySQL Support Team at Oracle.
However, I am happy to report that I have joined SkySQL!
So I’ll remain in the MySQL Community, and will still be providing support for MySQL.
I am grateful to everyone I’ve worked with and to everyone who has had a hand in improving MySQL (no matter how small or large) over the years! It’s been a blast working with such dedicated and passionate people, and I look forward to many more years of the same now at SkySQL! ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 17:32:31 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:996:"As some of you already know, I too have recently left the MySQL Support Team at Oracle.
However, I am happy to report that I have joined SkySQL!
So I’ll remain in the MySQL Community, and will still be providing support for MySQL. 
I am grateful to everyone I’ve worked with and to everyone who has had a hand in improving MySQL (no matter how small or large) over the years! It’s been a blast working with such dedicated and passionate people, and I look forward to many more years of the same now at SkySQL!
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Chris Calender";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:18;a:6:{s:4:"data";s:58:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:31:"Hack MySQL is longer maintained";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://hackmysql.com/blog/?p=145";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:69:"http://hackmysql.com/blog/2010/11/19/hack-mysql-is-longer-maintained/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:715:"I have decided to “officially” retire Hack MySQL, which is to say that the web site (pages, tools, etc.) will no longer be maintained. The MySQL universe has changed significantly since 2004 when I began writing tools. Similarly, my professional and personal lives have changed, too. I don’t think Hack MySQL is needed or relevant any longer, and I certainly do not have the time to maintain it.
The web site will remain online for perhaps another year (since Media Temple has always provided free hosting for this site–thanks!), then I will remove it or archive it somewhere else.
For further MySQL resources, see Planet MySQL. That aggregate is, imho, the pulse of the MySQL community.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 16:26:20 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:16:"Site Maintenance";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:12:"Media Temple";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Planet MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"retired";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1026:"I have decided to “officially” retire Hack MySQL, which is to say that the web site (pages, tools, etc.) will no longer be maintained. The MySQL universe has changed significantly since 2004 when I began writing tools. Similarly, my professional and personal lives have changed, too. I don’t think Hack MySQL is needed or relevant any longer, and I certainly do not have the time to maintain it.
The web site will remain online for perhaps another year (since Media Temple has always provided free hosting for this site–thanks!), then I will remove it or archive it somewhere else.
For further MySQL resources, see Planet MySQL. That aggregate is, imho, the pulse of the MySQL community.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Daniel Nichter";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:19;a:6:{s:4:"data";s:38:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:32:"MyMonitor 1.1 released. Finally!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:70:"tag:blogger.com,1999:blog-9144505959002328789.post-9103306582640862861";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:82:"http://karlssonondatabases.blogspot.com/2010/11/mymonitor-11-released-finally.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1536:"I know some people have used mymonitor on and off over the years, but it was a long ago since I created it, and things has moved on and changed. Among the things that didn't work well was the build process, and the fact that there were a few bugs in the normalization of SQL statements. Now this is fixed.If you have NOT used MyMonitor, it is a small ncurses based tool for monitoring SQL statements in your server, and work by polling the PROCESSLIST and the normalizing and sorting and grouping the result. What is "normalizing" then? It's the term I use to describe the process where a statement has all the literals replaced with ? in them, so I can see that:SELECT * FROM mytable WHERE id = 861is actually the same statement asSELECT * FROM mytable WHERE id = 387And by the way, this is also the same statement asSELECT * FROM mytable WHERE id = 177MyMonitor can look like this in action:You get the point I think. MyMonitor has a few more tricks up it's sleeve, like the ability to write out the monitored statements to a file and to do some simple timing on statements. All in all, this is a simple tool that can be real helpful at times. Many times less powerful than, say, MySQL Enterprise Monitor or MONYog, but still useful.If you are wondering: No GUI needed. What IS needed though is ncurses. And Windows support? Nope, but have a look at my MyQuery project instead, it has many more features, including staement monitoring.Download mymonitor from Sourceforge here: http://sourceforge.net/projects/mymonitor/Cheers/Karlsson";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 16:14:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2489:"I know some people have used mymonitor on and off over the years, but it was a long ago since I created it, and things has moved on and changed. Among the things that didn't work well was the build process, and the fact that there were a few bugs in the normalization of SQL statements. Now this is fixed.
If you have NOT used MyMonitor, it is a small ncurses based tool for monitoring SQL statements in your server, and work by polling the PROCESSLIST and the normalizing and sorting and grouping the result. What is "normalizing" then? It's the term I use to describe the process where a statement has all the literals replaced with ? in them, so I can see that:
SELECT * FROM mytable WHERE id = 861
is actually the same statement as
SELECT * FROM mytable WHERE id = 387
And by the way, this is also the same statement as
SELECT * FROM mytable WHERE id = 177
MyMonitor can look like this in action:
You get the point I think. MyMonitor has a few more tricks up it's sleeve, like the ability to write out the monitored statements to a file and to do some simple timing on statements. All in all, this is a simple tool that can be real helpful at times. Many times less powerful than, say, MySQL Enterprise Monitor or MONYog, but still useful.
If you are wondering: No GUI needed. What IS needed though is ncurses. And Windows support? Nope, but have a look at my MyQuery project instead, it has many more features, including staement monitoring.
Download mymonitor from Sourceforge here: http://sourceforge.net/projects/mymonitor/
Cheers
/Karlsson
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Anders Karlsson";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:20;a:6:{s:4:"data";s:58:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:44:"How to create a private cloud in your laptop";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-16959946.post-6180621817315153146";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:80:"http://datacharmer.blogspot.com/2010/11/how-to-create-private-cloud-in-your.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:11576:"Everybody is moving to cloud architectures, although not all agree on what cloud computing is. In my limited understanding, and for the purpose of my work, cloud computing is a quick and versatile availability of virtual machines. Now, if my purpose was deploying these machines, a private cloud in one host (namely, my laptop) would not make sense. But to create a flexible testing environment, it works very well. Users of virtual machines software such as VMWare or VirtualBox may ask what's new here. You can create many virtual machines and use them within the same host.True, but creating a new virtual machine in one minute without duplication of resources is not so easy. This is what this article covers. More specifically, it covers how to clone virtual machines efficiently with VMWare Fusion on a Mac OSX laptop.I have been a VMWare user for more than 10 years, mostly on Linux, where I used VMWare Workstation. One of the features that I liked on that application was the ability of cloning virtual machines, where the clones share the disk with the original VM. VMWare Fusion does not have a cloning feature, but there are ways of achieving the same result.If you copy a virtual machine and then open it with Fusion, it will ask you if you have copied it, and after your answer, it will use the new virtual machine independently from the first one. The only problem is that you will have twice as much disk space occupied. So, in addition to the space that will eventually got eaten up in your limited laptop, the duplication process is slow (copying 10 GB does not happen instantly). The method shown here will instead allow you to clone a virtual machine in less than one minute.Part I - How to clone a virtual machineI have got this recipe from HOWTO: Manual Linked Cloning in Fusion, a year old tutorial that I have integrated with updated images.Do it once - Create a virtual machine with everything you need. This is a once only step. Install a virtual machine as usual. In my case, I used Ubuntu server 10.10. When I said with everything you need I mean making sure that everything you think it will be necessary should be installed in this base VM. If you don't, you may end up installing the same thing in every cloned virtual machine.What I did was updating the software, then installing the build-essential package and the VMWare tools, and then enabling the "partner" repository to install the original Sun Java packages instead of the default OpenJDK. Next, I downloaded the source code for MySQL 5.5 and made sure that I had all the software necessary to compile and build the database.Finally, I updated the CPAN with my favorite modules, stopped the virtual machine, and I was ready for the next step.What I had in my hands was a working virtual machine. In order to make it clonable, I will go through some basic steps.Step 1. Copy the VM disksI create a new directory under the same path where the normal virtual machines are. I called it ub_srv_10_base, and I copied the .vmdk files into this new directory.Step 2. Modify the base disk to refer to fixed pathsIn each virtual machine, the disk that has the same name as the VM (without numerals) is a sort of index of the real files containing the data. This file needs to be edited for further usage. Originally, it looked like this:# Disk DescriptorFileversion=1encoding="UTF-8"CID=f88ac433parentCID=ffffffffisNativeSnapshot="no"createType="twoGbMaxExtentSparse"# Extent descriptionRW 4192256 SPARSE "ubuntu_server_10-s001.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s002.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s003.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s004.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s005.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s006.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s007.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s008.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s009.vmdk"RW 4192256 SPARSE "ubuntu_server_10-s010.vmdk"RW 20480 SPARSE "ubuntu_server_10-s011.vmdk"# The Disk Data Base #DDBddb.toolsVersion = "8323"ddb.virtualHWVersion = "7"ddb.longContentID = "d14a2f23de35287969b8eaebf88ac433"ddb.uuid = "60 00 C2 98 72 3b 6e 76-ff 05 b7 ff 8a 07 6e 92"ddb.geometry.cylinders = "2610"ddb.geometry.heads = "255"ddb.geometry.sectors = "63"ddb.adapterType = "lsilogic"To make it usable, I added a relative path to the file names, and removed the UUID.# Disk DescriptorFileversion=1encoding="UTF-8"CID=f88ac433parentCID=ffffffffisNativeSnapshot="no"createType="twoGbMaxExtentSparse"# Extent descriptionRW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s001.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s002.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s003.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s004.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s005.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s006.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s007.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s008.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s009.vmdk"RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s010.vmdk"RW 20480 SPARSE "../ub_srv_10_base/ubuntu_server_10-s011.vmdk"# The Disk Data Base #DDBddb.toolsVersion = "8323"ddb.virtualHWVersion = "7"ddb.longContentID = "d14a2f23de35287969b8eaebf88ac433"# ddb.uuid = "60 00 C2 98 72 3b 6e 76-ff 05 b7 ff 8a 07 6e 92"ddb.geometry.cylinders = "2610"ddb.geometry.heads = "255"ddb.geometry.sectors = "63"ddb.adapterType = "lsilogic"Now the file is ready to be used by other virtual machines. Notice that the files in this directory do not make a virtual machine. Only the storage part is here.Step 3. Make the disks read-onlyWe need to make sure that the base disks are not modified. So, we remove the "w" attribute from all the files in the base directory.$ chmod a-w ub_srv_base/*s0*Step 4. Remove the original virtual machineMake a copy if you want. And actually having a backup of the base disk directory is a very good idea. Compress it and save it to a good location. But remove the virtual machine from Fusion. This will avoid confusion later on.Do it many times - Create a cloneThis process looks frightfully long, especially if you count the images below, but in reality it's very quick and painless. Once you get familiar with it, you will be cloning virtual machines in no time at all.Back in VMWare Fusion, select "create a new Virtual Machine".Make sure that you don't have the Ubuntu CD in your read DVD player. Click on "continue without disk".Ignore the top options, and select "create a custom virtual machine".Select the same operating system that was used for your base VM.When the summary shows up, choose "Customize settings".Save the VM with a sensible name. Since you will be creating many of the same kind, choose a pattern that you can recognize. In this case, I used the base name with an additional letter at the end to identify the server.The disk is your main concern. The default settings want to create a 20 GB disk. Simply remove it.When asked if you want to get rid of the disk completely, say yes, "move it to the trash".Now, Very important, before you go to the next step. Use the command line, or a GUI, and copy the index .vmdk file from the base directory to the new virtual machine. I call it nd.vmdk (nd=new disk) but you can call it whatever you want. make sure that this file is not write protected.Add another disk. Here's the first trick part. By default, Fusion wants to recreate the same disk that you have just removed. But notice that there is a drop down menu on the right side of the panel. Click on "choose existing disk".Here you select the file that you have copied from the base directory. And now another tricky point. Make sure that you click on "Share this virtual disk ...", otherwise VMWare will make a copy of your original disk files.You can now change other parameters in the virtual machine, such as the amount of RAM and how many processors you want to use for it. I recommend unchecking the "connected" box next to the CDROM, to avoid a warning when more than a cloned VM work at the same time. You can always enable the CD later if you need it.Now Don't switch on your cloned virtual machine just yet. If you do, you get an incomprehensible message, where the VM complains about not being able to access the new disk file, while in reality it can't access the disks referred by that index file. This is where the precaution of write protecting the files comes handy. If you hadn't done that, the VM would access (and eventually modify) the virtual disk files, and possibly corrupt them. Instead, you need another step before having a functional clone.You need to create a snapshot. Once you have done that, the VM will write to the snapshot files all the deltas between your read-only disks and your VM final status.You can call the snapshot whatever you like.Finally, you can run the virtual machine. If other virtual machines are running, it may warn you that some devices might not be available. Ignore this warning unless you know for sure that there is a unique resource that should not be shared (usually, there isn't).Your virtual machine is ready to run. If you need to create three identical servers to simulate a cluster, and the original VM has 4GB of occupied storage, the operation won't cost you 16 GB, but just a few dozen MB:$ du -sh ub_*/ 21M ub_srv_10_10a.vmwarevm/ 22M ub_srv_10_10b.vmwarevm/ 21M ub_srv_10_10c.vmwarevm/3.9G ub_srv_10_base/Part II - Use your virtual machines from the command lineAlthough VMWare Fusion is handy for creating virtual machines, and for using GUI-based operating systems, it is less than desirable when you have several virtual machines with only text interface, and you want to use them from the command line, the same way you would do in most real life administration or QA operations.No need for further hacking, in this case. You can manage your virtual machines from the command line, using an utility called vmrun, which is located under /Library/Application Support/VMware Fusion/.You can read the full manual in a lengthy PDF document that is available online ( Using vmrun to Control Virtual Machines), but here's the short story.vmrun can start and stop a virtual machine. You just need to use it when the VMWare Fusion application is closed. For example:$ vmrun start $HOME/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx nogui2010-11-19 14:58:49 no printer configured or none available2010-11-19 14:58:49 adaptor daemon booted2010-11-19 14:58:49 connector "vmlocal" bootedAs part of my preparation of the virtual machine, I created a simple script that runs ifconfig and filters out the virtual machine IP to a file in the user's home.Using this information, I can then run the following commands:$ vmrun -gu qa -gp SECRETPWD runProgramInGuest \ $HOME/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx \ /home/qa/bin/get_ip$ vmrun -gu qa -gp SECRETPWD copyFileFromGuestToHost \ $HOME/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx \ /home/qa/myip ./server_a_ip$ cat server_a_ip 192.168.235.144This is just a simple example of what you can do. Once you have the IP address (which the DHCP server could change), you can connect to your virtual machine via ssh and do what you need. When you have finished, you can switch off the VM, again without need of using the GUI:$ vmrun stop ~/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx 2010-11-19 15:11:25 adaptor daemon shut down2010-11-19 15:11:25 connector "vmlocal" shut downHappy (cloud) hacking!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 14:19:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:2:"qa";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"laptop";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"cloud";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:14:"virtualization";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:16789:" | Everybody is moving to cloud architectures, although not all agree on what cloud computing is. In my limited understanding, and for the purpose of my work, cloud computing is a quick and versatile availability of virtual machines.
Now, if my purpose was deploying these machines, a private cloud in one host (namely, my laptop) would not make sense. But to create a flexible testing environment, it works very well.
Users of virtual machines software such as VMWare or VirtualBox may ask what's new here. You can create many virtual machines and use them within the same host.
True, but creating a new virtual machine in one minute without duplication of resources is not so easy. This is what this article covers. More specifically, it covers how to clone virtual machines efficiently with VMWare Fusion on a Mac OSX laptop. |
I have been a VMWare user for more than 10 years, mostly on Linux, where I used VMWare Workstation. One of the features that I liked on that application was the ability of cloning virtual machines, where the clones share the disk with the original VM. VMWare Fusion does not have a cloning feature, but there are ways of achieving the same result.
If you copy a virtual machine and then open it with Fusion, it will ask you if you have copied it, and after your answer, it will use the new virtual machine independently from the first one. The only problem is that you will have twice as much disk space occupied. So, in addition to the space that will eventually got eaten up in your limited laptop, the duplication process is slow (copying 10 GB does not happen instantly). The method shown here will instead allow you to clone a virtual machine in less than one minute.
Part I - How to clone a virtual machine
I have got this recipe from HOWTO: Manual Linked Cloning in Fusion, a year old tutorial that I have integrated with updated images.
Do it once - Create a virtual machine with everything you need.
This is a once only step. Install a virtual machine as usual. In my case, I used Ubuntu server 10.10. When I said with everything you need I mean making sure that everything you think it will be necessary should be installed in this base VM. If you don't, you may end up installing the same thing in every cloned virtual machine.
What I did was updating the software, then installing the build-essential package and the VMWare tools, and then enabling the "partner" repository to install the original Sun Java packages instead of the default OpenJDK. Next, I downloaded the source code for MySQL 5.5 and made sure that I had all the software necessary to compile and build the database.
Finally, I updated the CPAN with my favorite modules, stopped the virtual machine, and I was ready for the next step.
What I had in my hands was a working virtual machine. In order to make it clonable, I will go through some basic steps.
Step 1. Copy the VM disks
I create a new directory under the same path where the normal virtual machines are. I called it ub_srv_10_base, and I copied the .vmdk files into this new directory.
Step 2. Modify the base disk to refer to fixed paths
In each virtual machine, the disk that has the same name as the VM (without numerals) is a sort of index of the real files containing the data. This file needs to be edited for further usage. Originally, it looked like this:
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=f88ac433
parentCID=ffffffff
isNativeSnapshot="no"
createType="twoGbMaxExtentSparse"
# Extent description
RW 4192256 SPARSE "ubuntu_server_10-s001.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s002.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s003.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s004.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s005.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s006.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s007.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s008.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s009.vmdk"
RW 4192256 SPARSE "ubuntu_server_10-s010.vmdk"
RW 20480 SPARSE "ubuntu_server_10-s011.vmdk"
# The Disk Data Base
#DDB
ddb.toolsVersion = "8323"
ddb.virtualHWVersion = "7"
ddb.longContentID = "d14a2f23de35287969b8eaebf88ac433"
ddb.uuid = "60 00 C2 98 72 3b 6e 76-ff 05 b7 ff 8a 07 6e 92"
ddb.geometry.cylinders = "2610"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"
To make it usable, I added a relative path to the file names, and removed the UUID.
# Disk DescriptorFile
version=1
encoding="UTF-8"
CID=f88ac433
parentCID=ffffffff
isNativeSnapshot="no"
createType="twoGbMaxExtentSparse"
# Extent description
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s001.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s002.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s003.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s004.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s005.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s006.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s007.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s008.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s009.vmdk"
RW 4192256 SPARSE "../ub_srv_10_base/ubuntu_server_10-s010.vmdk"
RW 20480 SPARSE "../ub_srv_10_base/ubuntu_server_10-s011.vmdk"
# The Disk Data Base
#DDB
ddb.toolsVersion = "8323"
ddb.virtualHWVersion = "7"
ddb.longContentID = "d14a2f23de35287969b8eaebf88ac433"
# ddb.uuid = "60 00 C2 98 72 3b 6e 76-ff 05 b7 ff 8a 07 6e 92"
ddb.geometry.cylinders = "2610"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"
Now the file is ready to be used by other virtual machines. Notice that the files in this directory do not make a virtual machine. Only the storage part is here.
Step 3. Make the disks read-only
We need to make sure that the base disks are not modified. So, we remove the "w" attribute from all the files in the base directory.
$ chmod a-w ub_srv_base/*s0*
Step 4. Remove the original virtual machine
Make a copy if you want. And actually having a backup of the base disk directory is a very good idea. Compress it and save it to a good location. But remove the virtual machine from Fusion. This will avoid confusion later on.
Do it many times - Create a clone
This process looks frightfully long, especially if you count the images below, but in reality it's very quick and painless. Once you get familiar with it, you will be cloning virtual machines in no time at all.
Back in VMWare Fusion, select "create a new Virtual Machine".
Make sure that you don't have the Ubuntu CD in your read DVD player. Click on "continue without disk".
Ignore the top options, and select "create a custom virtual machine".
Select the same operating system that was used for your base VM.
When the summary shows up, choose "Customize settings".
Save the VM with a sensible name. Since you will be creating many of the same kind, choose a pattern that you can recognize. In this case, I used the base name with an additional letter at the end to identify the server.
The disk is your main concern. The default settings want to create a 20 GB disk. Simply remove it.
When asked if you want to get rid of the disk completely, say yes, "move it to the trash".
Now, Very important, before you go to the next step. Use the command line, or a GUI, and copy the index .vmdk file from the base directory to the new virtual machine. I call it nd.vmdk (nd=new disk) but you can call it whatever you want. make sure that this file is not write protected.
|
Add another disk. Here's the first trick part. By default, Fusion wants to recreate the same disk that you have just removed. But notice that there is a drop down menu on the right side of the panel. Click on "choose existing disk".
Here you select the file that you have copied from the base directory. And now another tricky point. Make sure that you click on "Share this virtual disk ...", otherwise VMWare will make a copy of your original disk files.
You can now change other parameters in the virtual machine, such as the amount of RAM and how many processors you want to use for it. I recommend unchecking the "connected" box next to the CDROM, to avoid a warning when more than a cloned VM work at the same time. You can always enable the CD later if you need it.
Now Don't switch on your cloned virtual machine just yet. If you do, you get an incomprehensible message, where the VM complains about not being able to access the new disk file, while in reality it can't access the disks referred by that index file. This is where the precaution of write protecting the files comes handy. If you hadn't done that, the VM would access (and eventually modify) the virtual disk files, and possibly corrupt them. Instead, you need another step before having a functional clone.
You need to create a snapshot. Once you have done that, the VM will write to the snapshot files all the deltas between your read-only disks and your VM final status.
You can call the snapshot whatever you like.
Finally, you can run the virtual machine. If other virtual machines are running, it may warn you that some devices might not be available. Ignore this warning unless you know for sure that there is a unique resource that should not be shared (usually, there isn't).
Your virtual machine is ready to run. If you need to create three identical servers to simulate a cluster, and the original VM has 4GB of occupied storage, the operation won't cost you 16 GB, but just a few dozen MB:
$ du -sh ub_*/
21M ub_srv_10_10a.vmwarevm/
22M ub_srv_10_10b.vmwarevm/
21M ub_srv_10_10c.vmwarevm/
3.9G ub_srv_10_base/
Part II - Use your virtual machines from the command line
Although VMWare Fusion is handy for creating virtual machines, and for using GUI-based operating systems, it is less than desirable when you have several virtual machines with only text interface, and you want to use them from the command line, the same way you would do in most real life administration or QA operations.
No need for further hacking, in this case. You can manage your virtual machines from the command line, using an utility called vmrun, which is located under /Library/Application Support/VMware Fusion/.
You can read the full manual in a lengthy PDF document that is available online ( Using vmrun to Control Virtual Machines), but here's the short story.
vmrun can start and stop a virtual machine. You just need to use it when the VMWare Fusion application is closed. For example:
$ vmrun start $HOME/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx nogui
2010-11-19 14:58:49 no printer configured or none available
2010-11-19 14:58:49 adaptor daemon booted
2010-11-19 14:58:49 connector "vmlocal" booted
As part of my preparation of the virtual machine, I created a simple script that runs ifconfig and filters out the virtual machine IP to a file in the user's home.
Using this information, I can then run the following commands:
$ vmrun -gu qa -gp SECRETPWD runProgramInGuest \
$HOME/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx \
/home/qa/bin/get_ip
$ vmrun -gu qa -gp SECRETPWD copyFileFromGuestToHost \
$HOME/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx \
/home/qa/myip ./server_a_ip
$ cat server_a_ip
192.168.235.144
This is just a simple example of what you can do. Once you have the IP address (which the DHCP server could change), you can connect to your virtual machine via ssh and do what you need. When you have finished, you can switch off the VM, again without need of using the GUI:
$ vmrun stop ~/vmware/vm/ub_srv_10_10a.vmwarevm/ub_srv_10_10a.vmx 2010-11-19 15:11:25 adaptor daemon shut down
2010-11-19 15:11:25 connector "vmlocal" shut down
Happy (cloud) hacking!
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Giuseppe Maxia";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:21;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:84:"Customization of monitors & advisors become more easy with the latest MONyog 4.1";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:34:"http://www.webyog.com/blog/?p=2254";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:117:"http://www.webyog.com/blog/2010/11/19/customization-of-monitors-advisors-become-more-easy-with-the-latest-monyog-4-1/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1990:"The highlights of this release are:
* A brand new helper function interface: We have implemented an easy-form based interface for customizing helper functions. Any customization of helper functions or user defined functions being used with earlier versions have to be migrated manually to this version. From this version and onwards a GUI-’conflict resolver’ will guide the users very similar to how it works for customized counters.
* Apply settings to all servers of a specific tag: MONyog now has an option to apply a group of settings to all the servers with a specific tag. (Please note: New servers registered later with the same tag are not considered)
* Enhanced readability of MONyog logs: MONyog logs from now on contain the server name along with error making it easy to backtrack.
* Earlier multiple alerts for same issue would be sent if the counter was alertable for more than the specified number of data collections. Now we record what alerts have already been sent and will only send once. Settings in register server page has been updated accordingly.
Minor features & fixes:
* Automatic cleaning of size zero dump files: We have automated deleting dump files of size zero (that could be left behind if MONyog was terminated abruptly).
* Added an option to change the refresh interval from the processlist page.
* Processlist-based sniffer will no longer show the query ‘SHOW FULL PROCESSLIST’ executed by MONyog (Note: This only applies to the processlist-based sniffer and not to the processlist page where user has configurable filtering options).
* Fixed a (SQLite) database lock resulting in failure to update Dashboards and Monitors/Advisors pages.
* When SFTP option was chosen to analyze ‘MySQL general query log’ and SSH was disabled, MONyog still used SFTP to analyze the logs with out throwing any error.
* Small GUI enhancement/improvements.
Downloads: http://webyog.com/en/downloads.php
Purchase: http://webyog.com/en/buy.php";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 14:15:12 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:6:"MONyog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"Releases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2528:"The highlights of this release are:
* A brand new helper function interface: We have implemented an easy-form based interface for customizing helper functions. Any customization of helper functions or user defined functions being used with earlier versions have to be migrated manually to this version. From this version and onwards a GUI-’conflict resolver’ will guide the users very similar to how it works for customized counters.
* Apply settings to all servers of a specific tag: MONyog now has an option to apply a group of settings to all the servers with a specific tag. (Please note: New servers registered later with the same tag are not considered)
* Enhanced readability of MONyog logs: MONyog logs from now on contain the server name along with error making it easy to backtrack.
* Earlier multiple alerts for same issue would be sent if the counter was alertable for more than the specified number of data collections. Now we record what alerts have already been sent and will only send once. Settings in register server page has been updated accordingly.
Minor features & fixes:
* Automatic cleaning of size zero dump files: We have automated deleting dump files of size zero (that could be left behind if MONyog was terminated abruptly).
* Added an option to change the refresh interval from the processlist page.
* Processlist-based sniffer will no longer show the query ‘SHOW FULL PROCESSLIST’ executed by MONyog (Note: This only applies to the processlist-based sniffer and not to the processlist page where user has configurable filtering options).
* Fixed a (SQLite) database lock resulting in failure to update Dashboards and Monitors/Advisors pages.
* When SFTP option was chosen to analyze ‘MySQL general query log’ and SSH was disabled, MONyog still used SFTP to analyze the logs with out throwing any error.
* Small GUI enhancement/improvements.
Downloads: http://webyog.com/en/downloads.php
Purchase: http://webyog.com/en/buy.php
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:6:"Webyog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:22;a:6:{s:4:"data";s:38:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:70:"Exporting / importing in the Pentaho Data Integration 4.0.1 Repository";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:58:"tag:blogger.com,1999:blog-29711306.post-205801357897112531";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:70:"http://www.johndz.com/2010/11/exporting-importing-in-pentaho-data.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2074:"After upgrading from PDI 3.x to 4.0.1, I've found that the old way of exporting an repository subfolder in a single step is gone. It's still important for me to be able to do this since I use separate repositories for Development, Test, and Production, and these systems must remain isolated.As a work around, here is what I am doing for now:You can still keep your jobs and transforms organized in repository folders/sub-folders. But you have to now export each (open) job and transformation individually using File -> Export -> To XMLI just save these files to an empty temporary directory on the file system.Then I disconnect from the source ( development) repository and connect to the destination repository, ( ie. Test or Production).In there, use :Tools -> Repository -> Import RepositoryIn the dialog that displays:Navigate to your temporary file system directory, and select all files you want to import. You can select more then one / all files by holding the shift key,much like selecting multiple rows in a spreadsheet.Click on the "Open" buttonThis will import your jobs and transformations at once. It will even create your sub-folders in the repository if they don't exist.---------------------------------------------------The first step, of course, is to backup your repository. My repositories use MySQL databases, so here is the form of the command I use:mysqldump --user=youruser --host=your.host --password=your.password --skip-opt --add-locks --create-options --databases DATABASENAME > backupfilename.out Between release cycles, I verify my backups by using them to restore/refresh my Development and Test repositories from Production.---------------------------------------------------I attended the Pentaho Agile-BI tour in Denver this week and met some other folks using PDI and the BI Suite, which was great. One of the discussions some of us had was about forming a Pentaho user group in the Denver / Boulder Area. If anyone is interested look me up on LinkedIn.com or send me an email and lets at least start an online discussion group.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 11:58:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2648:"After upgrading from PDI 3.x to 4.0.1, I've found that the old way of exporting an repository subfolder in a single step is gone. It's still important for me to be able to do this since I use separate repositories for Development, Test, and Production, and these systems must remain isolated.
As a work around, here is what I am doing for now:
You can still keep your jobs and transforms organized in repository folders/sub-folders. But you have to now export each (open) job and transformation individually using File -> Export -> To XML
I just save these files to an empty temporary directory on the file system.
Then I disconnect from the source ( development) repository and connect to the destination repository, ( ie. Test or Production).
In there, use :
Tools -> Repository -> Import Repository
In the dialog that displays:
Navigate to your temporary file system directory, and select all files you want to import. You can select more then one / all files by holding the shift key,
much like selecting multiple rows in a spreadsheet.
Click on the "Open" button
This will import your jobs and transformations at once. It will even create your sub-folders in the repository if they don't exist.
---------------------------------------------------
The first step, of course, is to backup your repository. My repositories use MySQL databases, so here is the form of the command I use:
mysqldump --user=youruser --host=your.host --password=your.password --skip-opt --add-locks --create-options --databases DATABASENAME > backupfilename.out
Between release cycles, I verify my backups by using them to restore/refresh my Development and Test repositories from Production.
---------------------------------------------------
I attended the Pentaho Agile-BI tour in Denver this week and met some other folks using PDI and the BI Suite, which was great. One of the discussions some of us had was about forming a Pentaho user group in the Denver / Boulder Area. If anyone is interested look me up on LinkedIn.com or send me an email and lets at least start an online discussion group.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"John Dzilvelis";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:23;a:6:{s:4:"data";s:48:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:58:"European Lunchtime Webinar: MySQL - Top 5 Performance Sins";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:44:"tag:blogs.oracle.com,2010:/mysql//3342.23778";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:100:"http://blogs.oracle.com/mysql/2010/11/european_lunchtime_webinar_mysql_-_top_5_performance_sins.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:616:"Next
Thursday Oracle University will be hosting a lunchtime webinar, between 13.00
and 14.00 CET, to help you better understand the common MySQL mistakes impacting
performance that you should avoid.
Oracle's
senior MySQL instructor and consultant Kai Voigt will review the MySQL "no no's"
and advise on what simple solutions should be used instead. Kai will also
provide an introduction to indexing, rewriting of queries, monitoring
the use of temporary tables, and other things you should know about.
You can register here
for this free webinar taking place on Thursday November 25th at
13.00 CET.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 11:09:19 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:17:"MySQL Performance";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1115:"Next
Thursday Oracle University will be hosting a lunchtime webinar, between 13.00
and 14.00 CET, to help you better understand the common MySQL mistakes impacting
performance that you should avoid.
Oracle's
senior MySQL instructor and consultant Kai Voigt will review the MySQL "no no's"
and advise on what simple solutions should be used instead. Kai will also
provide an introduction to indexing, rewriting of queries, monitoring
the use of temporary tables, and other things you should know about.
You can register here
for this free webinar taking place on Thursday November 25th at
13.00 CET.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:18:"Oracle MySQL Group";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:24;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:76:"SQLyog MySQL GUI 8.7 Released – Form View, improved SQL Formatter and more";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:34:"http://www.webyog.com/blog/?p=2243";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:110:"http://www.webyog.com/blog/2010/11/19/sqlyog-mysql-gui-8-7-released-form-view-improved-sql-formatter-and-more/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1543:"We are pleased to announce this new major release of SQLyog MySQL GUI.
The major enhancements with this release are:
* Information in tabbed interface can be reordered.
Just like you will find it in all modern browsers, TABs can now be reordered using drag’n’drop in the GUI. The option is available for connection tabs as well as Query/Query Builder/Schema Designer tabs.
* Fast and convenient data entry from the keyboard.
We have added ‘FORM view’ option (in addition to ‘GRID view’ and ‘TEXT view’ already available) for managing data in DATA and RESULT tabs. ‘FORM view’ will display one row at a time and is in particular convenient when entering data from the keyboard.
* Query Formatter
The SQLyog Query Formatter is now completely rewritten. In principle all types of SQL statements are now supported and all limitations with the old implementation are lifted.
* Several editor enhancements including:
The Editor now has a ‘folding option’. Multi-line code and comment blocks between bracket-pairs, BEGIN-END pairs, C-style comment pairs (/* and */), WHILE-END WHILE pairs, LOOP-END LOOP pairs, REPEAT-END REPEAT pairs and CASE-END CASE pairs can now be ‘folded’ in/out (ie: hidden or unhidden).
Add to that numerous stability fixes, GUI improvements and more. Read full details here.
We hope you will like this new version – and do not hesitate to contact us if you have comments, suggestions or whatever.
Downloads: http://webyog.com/en/downloads.php
Purchase: http://webyog.com/en/buy.php";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 19 Nov 2010 10:01:12 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:8:"Releases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"SQLyog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2567:"We are pleased to announce this new major release of SQLyog MySQL GUI.
The major enhancements with this release are:
* Information in tabbed interface can be reordered.
Just like you will find it in all modern browsers, TABs can now be reordered using drag’n’drop in the GUI. The option is available for connection tabs as well as Query/Query Builder/Schema Designer tabs.
* Fast and convenient data entry from the keyboard.
We have added ‘FORM view’ option (in addition to ‘GRID view’ and ‘TEXT view’ already available) for managing data in DATA and RESULT tabs. ‘FORM view’ will display one row at a time and is in particular convenient when entering data from the keyboard.
* Query Formatter
The SQLyog Query Formatter is now completely rewritten. In principle all types of SQL statements are now supported and all limitations with the old implementation are lifted.
* Several editor enhancements including:
The Editor now has a ‘folding option’. Multi-line code and comment blocks between bracket-pairs, BEGIN-END pairs, C-style comment pairs (/* and */), WHILE-END WHILE pairs, LOOP-END LOOP pairs, REPEAT-END REPEAT pairs and CASE-END CASE pairs can now be ‘folded’ in/out (ie: hidden or unhidden).
Add to that numerous stability fixes, GUI improvements and more. Read full details here.
We hope you will like this new version – and do not hesitate to contact us if you have comments, suggestions or whatever.
Downloads: http://webyog.com/en/downloads.php
Purchase: http://webyog.com/en/buy.php
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:6:"Webyog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:25;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:10:"It Happens";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:27:"http://inaugust.com/post/93";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:27:"http://inaugust.com/post/93";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2101:"It's entirely possible, given my propensity to swear and rant and declare things to be Web Scale, that folks might get the idea that I do not like our newer NoSQL bretheren. Quite the contrary - I think almost all of them are exciting and well suited to various tasks. My biggest beef is in thinking that a single technology should be square-pegged into every round-hole anyone can find. This holds true for Drizzle and MySQL as well, by the way. Just because I work on Drizzle does not mean I think Drizzle should be the method of data storage for every blessed thing on the face of the planet - that would be silly!I bring this up because I had my first instance of thinking that Cassandra might be a good choice for a project someone was showing me yesterday. In this case, the project was collecting data about events, but it as the data was events about real-world interactions, the particular information stored about each collected event had the potential to be wildly different. It could have been modeled (eventually) in SQL, but it seemed much more likely that a column-store like Cassandra would be entirely more suitable.I was also considering CouchDB the other day as a choice for cue storage backing a show-control system I'm working on, because of the document storage model. In this case, I believe I will use Drizzle because integrating with the pluggable protocol system will allow me to just speak the Open Sound Control protocol directly to Drizzle, and that's actually more important in this case.Point being - when I rant about the suitability of NoSQL solutions, I'm mainly complaining that in many cases it seems to me that they're using them because they're popular or trendy and not because they are or are not actually suited to the task at hand. If they are suited to the task at hand, then by all means - use them. Heck- I've been convincing people to use NDB for years when it's the right fit.We don't need a winner in the data storage world - we need a vibrant culture out of which tools suitable for tasks can be selected. ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 18:59:23 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:6:"ubuntu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:7:"drizzle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2366:"It's entirely possible, given my propensity to swear and rant and declare things to be Web Scale, that folks might get the idea that I do not like our newer NoSQL bretheren. Quite the contrary - I think almost all of them are exciting and well suited to various tasks. My biggest beef is in thinking that a single technology should be square-pegged into every round-hole anyone can find. This holds true for Drizzle and MySQL as well, by the way. Just because I work on Drizzle does not mean I think Drizzle should be the method of data storage for every blessed thing on the face of the planet - that would be silly!
I bring this up because I had my first instance of thinking that Cassandra might be a good choice for a project someone was showing me yesterday. In this case, the project was collecting data about events, but it as the data was events about real-world interactions, the particular information stored about each collected event had the potential to be wildly different. It could have been modeled (eventually) in SQL, but it seemed much more likely that a column-store like Cassandra would be entirely more suitable.
I was also considering CouchDB the other day as a choice for cue storage backing a show-control system I'm working on, because of the document storage model. In this case, I believe I will use Drizzle because integrating with the pluggable protocol system will allow me to just speak the Open Sound Control protocol directly to Drizzle, and that's actually more important in this case.
Point being - when I rant about the suitability of NoSQL solutions, I'm mainly complaining that in many cases it seems to me that they're using them because they're popular or trendy and not because they are or are not actually suited to the task at hand. If they are suited to the task at hand, then by all means - use them. Heck- I've been convincing people to use NDB for years when it's the right fit.
We don't need a winner in the data storage world - we need a vibrant culture out of which tools suitable for tasks can be selected.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Monty Taylor";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:26;a:6:{s:4:"data";s:63:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:45:"Loose index scan vs. covered indexes in MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.arubin.org/blog/?p=58";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:83:"http://www.arubin.org/blog/2010/11/18/loose-index-scan-vs-covered-indexes-in-mysql/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:4442:"Loose index scan in MySQL can really help optimizing “group by” queries in some cases (for example, if you have only min() and/or max() as your aggregate functions). For example, if you have this query (to find maximum delay for all US flights with departure on Sundays in 2010):
select max(DepDelayMinutes), carrier, dayofweek
from ontime_2010
where dayofweek = 7
group by Carrier, dayofweek
the usual case will be adding a covered index on (dayofweek, Carrier, DepDelayMinutes). And MySQL will use this index fine (using index mean it will use the covered index):
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010
where dayofweek =7 group by Carrier, dayofweek\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: ref
possible_keys: covered
key: covered
key_len: 2
ref: const
rows: 905138
Extra: Using where; Using index
1 row in set (0.00 sec)
However, as the dayofweek part has low number of unique values, mysql will have to scan a lots of index entries (estimated rows: 905138).
MySQL can use loose index scan. Unfortunately, a o lots of limitations apply:
The query is over a single table.
The GROUP BY names only columns that form a leftmost prefix of the index and no other columns.
The only aggregate functions used in the select list (if any) are MIN() and MAX(), same column
etc… (see docs for details)
As our example query is suitable for loose index scan, we can create another index:
mysql> alter table ontime_2010 add key lis1(Carrier, dayofweek, DepDelayMinutes);
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010
where dayofweek =7 group by Carrier, dayofweek \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: range
possible_keys: DayOfWeek,covered
key: lis1
key_len: 5
ref: NULL
rows: 19
Extra: Using where; Using index for group-by
1 row in set (0.01 sec)
Here, Using index for group-by, means that MySQL uses loose index scan.
Also, and it is really great, it works with range on dayofweek too:
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010
where dayofweek > 3 group by Carrier, dayofweek \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: range
possible_keys: DayOfWeek,covered
key: lis1
key_len: 5
ref: NULL
rows: 19
Extra: Using where; Using index for group-by
1 row in set (0.00 sec)
And original covered index does not work with ranges in where clause:
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010 use index (covered)
where dayofweek > 3 group by Carrier, dayofweek \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: range
possible_keys: covered
key: covered
key_len: 2
ref: NULL
rows: 2416543
Extra: Using where; Using index; Using temporary; Using filesort
In the above example, MySQL uses index but still have to create temporary table and filesort.
Now speed comparison:
I’m using “ontime” flight performance statistics data from transtats.bts.gov
The table only consist of data for 2010.
Table size: 5M rows, ~2G in size. Table structure:
CREATE TABLE `ontime_2010` (
`YearD` int(11) DEFAULT NULL,
`MonthD` tinyint(4) DEFAULT NULL,
`DayofMonth` tinyint(4) DEFAULT NULL,
`DayOfWeek` tinyint(4) DEFAULT NULL,
`Carrier` char(2) DEFAULT NULL,
`Origin` char(5) DEFAULT NULL,
`DepDelayMinutes` int(11) DEFAULT NULL,
... more fields here ...
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Results (cached index and data):
“where dayofweek = 7″ (ref)
Loose index scan: 0 sec
Covered index: 0.6 sec
“where dayofweek > 3″ (range)
Loose index scan: 0 sec
index (+ filesort): 5.53 sec
I will also present the findings from this article (among other things) during the upcoming webinar, Getting the Best MySQL Performance in Your Products: Part 3, Query Tuning, which will take place on Tuesday, Nov 23 (webinar is free, webinar registration)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 17:51:24 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:18:"performance tuning";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:13:"covered index";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"index";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:16:"loose index scan";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:17:"mysql performance";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5569:"Loose index scan in MySQL can really help optimizing “group by” queries in some cases (for example, if you have only min() and/or max() as your aggregate functions). For example, if you have this query (to find maximum delay for all US flights with departure on Sundays in 2010):
select max(DepDelayMinutes), carrier, dayofweek
from ontime_2010
where dayofweek = 7
group by Carrier, dayofweek
the usual case will be adding a covered index on (dayofweek, Carrier, DepDelayMinutes). And MySQL will use this index fine (using index mean it will use the covered index):
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010
where dayofweek =7 group by Carrier, dayofweek\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: ref
possible_keys: covered
key: covered
key_len: 2
ref: const
rows: 905138
Extra: Using where; Using index
1 row in set (0.00 sec)
However, as the dayofweek part has low number of unique values, mysql will have to scan a lots of index entries (estimated rows: 905138).
MySQL can use loose index scan. Unfortunately, a o lots of limitations apply:
- The query is over a single table.
- The GROUP BY names only columns that form a leftmost prefix of the index and no other columns.
- The only aggregate functions used in the select list (if any) are MIN() and MAX(), same column
- etc… (see docs for details)
As our example query is suitable for loose index scan, we can create another index:
mysql> alter table ontime_2010 add key lis1(Carrier, dayofweek, DepDelayMinutes);
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010
where dayofweek =7 group by Carrier, dayofweek \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: range
possible_keys: DayOfWeek,covered
key: lis1
key_len: 5
ref: NULL
rows: 19
Extra: Using where; Using index for group-by
1 row in set (0.01 sec)
Here, Using index for group-by, means that MySQL uses loose index scan.
Also, and it is really great, it works with range on dayofweek too:
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010
where dayofweek > 3 group by Carrier, dayofweek \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: range
possible_keys: DayOfWeek,covered
key: lis1
key_len: 5
ref: NULL
rows: 19
Extra: Using where; Using index for group-by
1 row in set (0.00 sec)
And original covered index does not work with ranges in where clause:
mysql> explain select max(DepDelayMinutes), Carrier, dayofweek from ontime_2010 use index (covered)
where dayofweek > 3 group by Carrier, dayofweek \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: ontime_2010
type: range
possible_keys: covered
key: covered
key_len: 2
ref: NULL
rows: 2416543
Extra: Using where; Using index; Using temporary; Using filesort
In the above example, MySQL uses index but still have to create temporary table and filesort.
Now speed comparison:
I’m using “ontime” flight performance statistics data from transtats.bts.gov
The table only consist of data for 2010.
Table size: 5M rows, ~2G in size. Table structure:
CREATE TABLE `ontime_2010` (
`YearD` int(11) DEFAULT NULL,
`MonthD` tinyint(4) DEFAULT NULL,
`DayofMonth` tinyint(4) DEFAULT NULL,
`DayOfWeek` tinyint(4) DEFAULT NULL,
`Carrier` char(2) DEFAULT NULL,
`Origin` char(5) DEFAULT NULL,
`DepDelayMinutes` int(11) DEFAULT NULL,
... more fields here ...
) ENGINE=InnoDB DEFAULT CHARSET=latin1
Results (cached index and data):
“where dayofweek = 7″ (ref)
- Loose index scan: 0 sec
- Covered index: 0.6 sec
“where dayofweek > 3″ (range)
- Loose index scan: 0 sec
- index (+ filesort): 5.53 sec
I will also present the findings from this article (among other things) during the upcoming webinar, Getting the Best MySQL Performance in Your Products: Part 3, Query Tuning, which will take place on Tuesday, Nov 23 (webinar is free, webinar registration)
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Alexander Rubin";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:27;a:6:{s:4:"data";s:98:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:113:"TOTD #150: Collection of GlassFish, NetBeans, JPA, JSF, JAX-WS, EJB, Jersey, MySQL, Rails, Eclipse, and OSGi tips";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:69:"http://blogs.sun.com/arungupta/entry/totd_150_collection_of_glassfish";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:69:"http://blogs.sun.com/arungupta/entry/totd_150_collection_of_glassfish";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:11867:"This is the 150th tip published on this blog so decided to make it a collection of all the previous ones. Here is a tag cloud (created from wordle.net/create) from title of all the tips:
As expected GlassFish is the most prominent topic. And then there are several entries on NetBeans, JRuby/Rails, several Java EE 6 technologies like JPA, JAX-WS, JAX-RS, EJB, and JSF, and more entries on Eclipse, OSGi and some other tecnhologies too. Here is a complete collection of all the tips published so far:
#149: How to clean IntelliJ cache, preferences, etc on Mac OS X ?
#148: JPA2 Metamodel Classes in NetBeans 7.0 - Writing type-safe Criteria API
#147: Java Server Faces 2.0 Composite Components using NetBeans - DRY your code
#146: Understanding the EJB 3.1 Timer service in Java EE 6 - Programmatic, Deployment Descriptor, @Schedule
#145: CDI Events - a light-weight producer/consumer in Java EE 6
#144: CDI @Produces for container-managed @Resource
#143: Retrieve Twitter user timeline using using Jersey and OAuth
#142: GlassFish 3.1 - SSH Provisioning and Start/Stop instance/cluster on local/remote machines
#141: Running GlassFish 3.1 on Ubuntu 10.04 AMI on Amazon EC2
#140: Moving GlassFish Installation - Referenced file does not exist "osgi-main.jar"
#139: Asynchronous Request Processing using Servlets 3.0 and Java EE 6
#138: GlassFish 3.1 Milestone 1 - Clustering and Application Versioning Demos
#137: Asynchronous EJB, a light-weight JMS solution - Feature-rich Java EE 6
#136: Default Error Page using Servlets 3.0 - Improved productivity using Java EE 6
#135: JSF2 Composite Components using NetBeans IDE - lightweight Java EE 6
#134: Interceptors 1.1 in Java EE 6 - What and How ?
#133: JPA2 (JPQL & Criteria), JavaDB, and embedded GlassFish - perfect recipe for testing
#132: Servlets 3.0 in Embedded GlassFish Reloaded - lightweight Java EE 6
#131: Dynamic OSGi services in GlassFish - Using ServiceTracker
#130: Invoking a OSGi service from a JAX-WS Endpoint - OSGi and Enterprise Java
#129: Managed Beans 1.0 in Java EE 6 - What and How ?
#128: EJBContainer.createEJBContainer: Embedded EJB using GlassFish v3
#127: Embedding GlassFish in an existing OSGi runtime - Eclipse Equinox
#126: Creating an OSGi bundles using Eclipse and deploying in GlassFish
#125: Creating an OSGi bundles using NetBeans and deploying in GlassFish
#124: OSGi Declarative Services in GlassFish - Accessed from a Java EE client
#124: Using CDI + JPA with JAX-RS and JAX-WS
#123: f:ajax, Bean Validation for JSF, CDI for JSF and JPA 2.0 Criteria API - all in one Java EE 6 sample application
#122: Creating a JPA Persistence Unit using NetBeans 6.8
#121: JDBC resource for MySQL and Oracle sample database in GlassFish v3
#120: Deployment Descriptor-free Java EE 6 application using JSF 2.0 + EJB 3.1 + Servlets 3.0
#119: Telnet to GlassFish v3 with NetBeans 6.8 - "Could not open connection to the host"
#118: Managing OSGi bundles in GlassFish v3 - asadmin, filesystem, telnet console, web browser, REST, osgish
#117: Invoke a JAX-WS Web service from a Rails app deployed in GlassFish
#116: GlassFish v3 Administration using JavaFX front-end - JNLP available
#115: GlassFish in Eclipse - Integrated Bundle, Install Stand-alone or Update Existing plugin
#114: How to enable Java Console in Mac OS X, Windows, ... ?
#113: JavaFX front-end for GlassFish v3 Administration - Using REST interface
#112: Exposing Oracle database tables as RESTful entities using JAX-RS, GlassFish, and NetBeans
#111: Rails Scaffold for a pre-existing table using Oracle and GlassFish
#110: JRuby on Rails application using Oracle on GlassFish
#109: How to convert a JSF managed bean to JSR 299 bean (Web Beans) ?
#108: Java EE 6 web application (JSF 2.0 + JPA 2.0 + EJB 3.1) using Oracle, NetBeans, and GlassFish
#107: Connect to Oracle database using NetBeans
#106: How to install Oracle Database 10g on Mac OS X (Intel) ?
TOTD #105: GlassFish v3 Monitoring – How to monitor a Rails app using asadmin, JavaScript, jConsole, REST ?
#104: Popular Ruby-on-Rails applications on GlassFish v3 – Redmine, Typo, Substruct
#103: GlassFish v3 with different OSGi runtimes – Felix, Equinox, and Knoplerfish
#102: Java EE 6 (Servlet 3.0 and EJB 3.1) wizards in Eclipse
#101: Applying Servlet 3.0/Java EE 6 “web-fragment.xml” to Lift – Deploy on GlassFish v3
#100: Getting Started with Scala Lift on GlassFish v3
#99: Creating a Java EE 6 application using MySQL, JPA 2.0 and Servlet 3.0 with GlassFish Tools Bundle for Eclipse
#98: Create a Metro JAX-WS Web service using GlassFish Tools Bundle for Eclipse
#97: GlassFish Plugin with Eclipse 3.5
#96: GlassFish v3 REST Interface to Monitoring and Management - JSON, XML, and HTML representations
#95: EJB 3.1 + Java Server Faces 2.0 + JPA 2.0 web application - Getting Started with Java EE 6 using NetBeans 6.8 M1 & GlassFish v3
#94: A simple Java Server Faces 2.0 + JPA 2.0 application - Getting Started with Java EE 6 using NetBeans 6.8 M1 & GlassFish v3
#93: Getting Started with Java EE 6 using NetBeans 6.8 M1 & GlassFish v3 - A simple Servlet 3.0 + JPA 2.0 app
#92: Session Failover for Rails applications running on GlassFish
#91: Applying Java EE 6 "web-fragment.xml" to Apache Wicket - Deploy on GlassFish v3
#90: Migrating from Wicket 1.3.x to 1.4 - "Couldn't load DiskPageStore index from file" error
#89: How to add pagination to an Apache Wicket application
#88: How add pagination to Rails - will_paginate
#87: How to fix the error undefined method `new' for "Rack::Lock":String caused by Warbler/JRuby-Rack ?
#86: Getting Started with Apache Wicket on GlassFish
#85: Getting Started with Django Applications on GlassFish v3
#84: Using Apache + mod_proxy_balancer to load balance Ruby-on-Rails running on GlassFish
#83: Eclipse Tools Bundle for GlassFish 1.0 - Now Available!
#82: Getting Started with Servlet 3.0 and EJB 3.1 in Java EE 6 using NetBeans 6.7
#81: How to use nginx to load balance a cluster of GlassFish Gem ?
#80: Sinatra CRUD application using Haml templates with JRuby and GlassFish Gem
#79: Getting Started with Sinatra applications on JRuby and GlassFish Gem
#78: GlassFish, EclipseLink, and MySQL efficient pagination using LIMIT
#77: Running Seam examples with GlassFish
#76: JRuby 1.2, Rails 2.3, GlassFish Gem 0.9.3, ActiveRecord JDBC Adapter 0.9.1 - can they work together ?
#75: Getting Started with Grails using GlassFish v3 Embedded
#74: JRuby and GlassFish Integration Test #5: JRuby 1.2.0 RC2 + Rails 2.x.x + GlassFish + Redmine
#73: JRuby and GlassFish Integration Test #4: JRuby 1.2.0 RC2 + Rails 2.2.x + GlassFish v2 + Warbler
#72: JRuby and GlassFish Integration Test #3: JRuby 1.2.0 RC2 + Rails 2.2.x + GlassFish v3
#71: JRuby and GlassFish Integration Test #2: JRuby 1.2.0 RC1 + Rails 2.2.x + GlassFish v3 Prelude
#70: JRuby and GlassFish Integration Test# 1: JRuby 1.2.0 RC1 + Rails 2.2.x + GlassFish Gem
#69: GlassFish High Availability/Clustering using Sun Web Server + Load Balancer Plugin on Windows Vista
#68: Installing Zones in Open Solaris 2008/11 on Virtual Box
#67: How to front-end a GlassFish Cluster with Apache + mod_jk on Mac OSX Leopard ?
#66: GlassFish Eclipse Plugin 1.0.16 - Install v3 Prelude from the IDE
#65: Windows 7 Beta 1 Build 7000 on Virtual Box: NetBeans + Rails + GlassFish + MySQL
#64: OpenSolaris 2008/11 using Virtual Box
#63: jmx4r gem - How to manage/monitor your Rails/Merb applications on JRuby/GlassFish ?
#62: How to remotely manage/monitor your Rails/Merb applications on JRuby/GlassFish using JMX API ?
#61: How to locally manage/monitor your Rails/Merb applications on JRuby/GlassFish using JMX ?
#60: Configure MySQL 6.0.x-alpha to NetBeans 6.5
#59: How to add Twitter feeds to blogs.sun.com ? + Other Twitter Tools
#58: Jersey and GlassFish - how to process POST requests ?
#57: Jersey Client API - simple and easy to use
#56: Simple RESTful Web service using Jersey and Embeddable GlassFish - Text and JSON output
#55: How to build GlassFish v3 Gem ?
#54: Java Server Faces with Eclipse IDE
#53: Scaffold in Merb using JRuby/GlassFish
#52: Getting Started with Merb using GlassFish Gem
#51: Embedding Google Maps in Java Server Faces using GMaps4JSF
#50: Mojarra 2.0 EDR2 is now available - Try them with GlassFish v3 and NetBeans 6.5
#49: Converting a JSF 1.2 application to JSF 2.0 - @ManagedBean
#48: Converting a JSF 1.2 application to JSF 2.0 - Facelets and Ajax
#47: Getting Started with Mojarra 2.0 nightly on GlassFish v2
#46: Facelets with Java Server Faces 1.2
#45: Ajaxifying Java Server Faces using JSF Extensions
#44: JDBC Connection Pooling for Rails on GlassFish v3
#43: GlassFish v3 Build Flavors
#42: Hello JavaServer Faces World with NetBeans and GlassFish
#41: How I created transparent logo of GlassFish using Gimp ?
#40: jQuery Autcomplete widget with MySQL, GlassFish, NetBeans
#39: Prototype/Script.aculo.us Autcomplete widget with MySQL, GlassFish, NetBeans
#38: Creating a MySQL Persistence Unit using NetBeans IDE
#37: SQLite3 with Ruby-on-Rails on GlassFish Gem
#36: Writing First Test for a Rails Application
#35: Rails Database Connection on Solaris
#34: Using Felix Shell with GlassFish
#33: Building GlassFish v3 Workspace
#32: Rails Deployment on GlassFish v3 from NetBeans IDE
#31: CRUD Application using Grails - Hosted on GlassFish and MySQL
#30: CRUD Application using Grails - Hosted on Jetty and HSQLDB
#29: Enabling "Available Plugins" tab in NetBeans IDE
#28: Getting Started with Rails 2.0 Scaffold
#27: Configurable Multiple Ruby Platforms in NetBeans 6.1 M1
#26: Overriding Database Defaults in Rails 2.0.2
#25: Rails application with PostgreSQL database using NetBeans
#24: Getting Started with Rails 2.0.x in JRuby 1.0.3 and JRuby 1.1RC1
#23: JavaFX Client invoking a Metro endpoint
#22: Java SE client for a Metro endpoint
#21: Metro 1.1 with GlassFish v2 UR1 and NetBeans 6
#20: How to create a new jMaki widget ?
#19: How to Add Metro Quality-of-Service to Contract-First Endpoint ?
#18: How to Build The GlassFish v3 Gem for JRuby ?
#17: Backing Up your Blog Posts on Roller
#16: Optimizing Metro Stubs by locally packaging the WSDL
#15: Delete/Update Row from Database using jMaki Data Table
#14: How to generate JRuby-on-Rails Controller on Windows (#9893)
#13: Setup Mongrel for JRuby-on-Rails applications on Windows
#12: Invoking a Java EE 5 Web service endpoint from JRuby
#11: Setup Mongrel cluster for JRuby-on-Rails applications on Unix
#10: Consuming JSON and XML representations generated by a Jersey endpoint in a jMaki Table widget
#9: Using JDBC connection pool/JNDI name from GlassFish in Rails Application
#8: Generating JSON using JAXB annotations in Jersey
#7: Switch between JRuby and CRuby interpreter in NetBeans 6
#6: Difference between Ruby Gem and Rails Plugin
#5: Loading data from beans in jMaki widgets
#4: How to convert a Session EJB to a Web service ?
#3: Using JavaDB with JRuby on Rails
#2: Change the endpoint address on a pre-generated Web services Stub
#1: SOAP Messaging Logging in Metro
Just for fun, here is another tag cloud:
You can access all the tips here. And keep those suggestions coming!
Technorati: totd glassfish netbeans jpa jsf jaxws jersey mysql rails osgi eclipse";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 14:00:23 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:12:{i:0;a:5:{s:4:"data";s:7:"General";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:7:"eclipse";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:9:"glassfish";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:5:"jaxws";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:6:"jersey";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:3:"jpa";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:3:"jsf";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:8;a:5:{s:4:"data";s:8:"netbeans";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:9;a:5:{s:4:"data";s:4:"osgi";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:10;a:5:{s:4:"data";s:5:"rails";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:11;a:5:{s:4:"data";s:4:"totd";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:26293:"This is the 150th tip published on this blog so decided to make it a collection of all the previous ones. Here is a tag cloud (created from wordle.net/create) from title of all the tips:
As expected GlassFish is the most prominent topic. And then there are several entries on NetBeans, JRuby/Rails, several Java EE 6 technologies like JPA, JAX-WS, JAX-RS, EJB, and JSF, and more entries on Eclipse, OSGi and some other tecnhologies too. Here is a complete collection of all the tips published so far:
- #149: How to clean IntelliJ cache, preferences, etc on Mac OS X ?
- #148: JPA2 Metamodel Classes in NetBeans 7.0 - Writing type-safe Criteria API
- #147: Java Server Faces 2.0 Composite Components using NetBeans - DRY your code
- #146: Understanding the EJB 3.1 Timer service in Java EE 6 - Programmatic, Deployment Descriptor, @Schedule
- #145: CDI Events - a light-weight producer/consumer in Java EE 6
- #144: CDI @Produces for container-managed @Resource
- #143: Retrieve Twitter user timeline using using Jersey and OAuth
- #142: GlassFish 3.1 - SSH Provisioning and Start/Stop instance/cluster on local/remote machines
- #141: Running GlassFish 3.1 on Ubuntu 10.04 AMI on Amazon EC2
- #140: Moving GlassFish Installation - Referenced file does not exist "osgi-main.jar"
- #139: Asynchronous Request Processing using Servlets 3.0 and Java EE 6
- #138: GlassFish 3.1 Milestone 1 - Clustering and Application Versioning Demos
- #137: Asynchronous EJB, a light-weight JMS solution - Feature-rich Java EE 6
- #136: Default Error Page using Servlets 3.0 - Improved productivity using Java EE 6
- #135: JSF2 Composite Components using NetBeans IDE - lightweight Java EE 6
- #134: Interceptors 1.1 in Java EE 6 - What and How ?
- #133: JPA2 (JPQL & Criteria), JavaDB, and embedded GlassFish - perfect recipe for testing
- #132: Servlets 3.0 in Embedded GlassFish Reloaded - lightweight Java EE 6
- #131: Dynamic OSGi services in GlassFish - Using ServiceTracker
- #130: Invoking a OSGi service from a JAX-WS Endpoint - OSGi and Enterprise Java
- #129: Managed Beans 1.0 in Java EE 6 - What and How ?
- #128: EJBContainer.createEJBContainer: Embedded EJB using GlassFish v3
- #127: Embedding GlassFish in an existing OSGi runtime - Eclipse Equinox
- #126: Creating an OSGi bundles using Eclipse and deploying in GlassFish
- #125: Creating an OSGi bundles using NetBeans and deploying in GlassFish
- #124: OSGi Declarative Services in GlassFish - Accessed from a Java EE client
- #124: Using CDI + JPA with JAX-RS and JAX-WS
- #123: f:ajax, Bean Validation for JSF, CDI for JSF and JPA 2.0 Criteria API - all in one Java EE 6 sample application
- #122: Creating a JPA Persistence Unit using NetBeans 6.8
- #121: JDBC resource for MySQL and Oracle sample database in GlassFish v3
- #120: Deployment Descriptor-free Java EE 6 application using JSF 2.0 + EJB 3.1 + Servlets 3.0
- #119: Telnet to GlassFish v3 with NetBeans 6.8 - "Could not open connection to the host"
- #118: Managing OSGi bundles in GlassFish v3 - asadmin, filesystem, telnet console, web browser, REST, osgish
- #117: Invoke a JAX-WS Web service from a Rails app deployed in GlassFish
- #116: GlassFish v3 Administration using JavaFX front-end - JNLP available
- #115: GlassFish in Eclipse - Integrated Bundle, Install Stand-alone or Update Existing plugin
- #114: How to enable Java Console in Mac OS X, Windows, ... ?
- #113: JavaFX front-end for GlassFish v3 Administration - Using REST interface
- #112: Exposing Oracle database tables as RESTful entities using JAX-RS, GlassFish, and NetBeans
- #111: Rails Scaffold for a pre-existing table using Oracle and GlassFish
- #110: JRuby on Rails application using Oracle on GlassFish
- #109: How to convert a JSF managed bean to JSR 299 bean (Web Beans) ?
- #108: Java EE 6 web application (JSF 2.0 + JPA 2.0 + EJB 3.1) using Oracle, NetBeans, and GlassFish
- #107: Connect to Oracle database using NetBeans
- #106: How to install Oracle Database 10g on Mac OS X (Intel) ?
- TOTD #105: GlassFish v3 Monitoring – How to monitor a Rails app using asadmin, JavaScript, jConsole, REST ?
- #104: Popular Ruby-on-Rails applications on GlassFish v3 – Redmine, Typo, Substruct
- #103: GlassFish v3 with different OSGi runtimes – Felix, Equinox, and Knoplerfish
- #102: Java EE 6 (Servlet 3.0 and EJB 3.1) wizards in Eclipse
- #101: Applying Servlet 3.0/Java EE 6 “web-fragment.xml” to Lift – Deploy on GlassFish v3
- #100: Getting Started with Scala Lift on GlassFish v3
- #99: Creating a Java EE 6 application using MySQL, JPA 2.0 and Servlet 3.0 with GlassFish Tools Bundle for Eclipse
- #98: Create a Metro JAX-WS Web service using GlassFish Tools Bundle for Eclipse
- #97: GlassFish Plugin with Eclipse 3.5
- #96: GlassFish v3 REST Interface to Monitoring and Management - JSON, XML, and HTML representations
- #95: EJB 3.1 + Java Server Faces 2.0 + JPA 2.0 web application - Getting Started with Java EE 6 using NetBeans 6.8 M1 & GlassFish v3
- #94: A simple Java Server Faces 2.0 + JPA 2.0 application - Getting Started with Java EE 6 using NetBeans 6.8 M1 & GlassFish v3
- #93: Getting Started with Java EE 6 using NetBeans 6.8 M1 & GlassFish v3 - A simple Servlet 3.0 + JPA 2.0 app
- #92: Session Failover for Rails applications running on GlassFish
- #91: Applying Java EE 6 "web-fragment.xml" to Apache Wicket - Deploy on GlassFish v3
- #90: Migrating from Wicket 1.3.x to 1.4 - "Couldn't load DiskPageStore index from file" error
- #89: How to add pagination to an Apache Wicket application
- #88: How add pagination to Rails - will_paginate
- #87: How to fix the error undefined method `new' for "Rack::Lock":String caused by Warbler/JRuby-Rack ?
- #86: Getting Started with Apache Wicket on GlassFish
- #85: Getting Started with Django Applications on GlassFish v3
- #84: Using Apache + mod_proxy_balancer to load balance Ruby-on-Rails running on GlassFish
- #83: Eclipse Tools Bundle for GlassFish 1.0 - Now Available!
- #82: Getting Started with Servlet 3.0 and EJB 3.1 in Java EE 6 using NetBeans 6.7
- #81: How to use nginx to load balance a cluster of GlassFish Gem ?
- #80: Sinatra CRUD application using Haml templates with JRuby and GlassFish Gem
- #79: Getting Started with Sinatra applications on JRuby and GlassFish Gem
- #78: GlassFish, EclipseLink, and MySQL efficient pagination using LIMIT
- #77: Running Seam examples with GlassFish
- #76: JRuby 1.2, Rails 2.3, GlassFish Gem 0.9.3, ActiveRecord JDBC Adapter 0.9.1 - can they work together ?
- #75: Getting Started with Grails using GlassFish v3 Embedded
- #74: JRuby and GlassFish Integration Test #5: JRuby 1.2.0 RC2 + Rails 2.x.x + GlassFish + Redmine
- #73: JRuby and GlassFish Integration Test #4: JRuby 1.2.0 RC2 + Rails 2.2.x + GlassFish v2 + Warbler
- #72: JRuby and GlassFish Integration Test #3: JRuby 1.2.0 RC2 + Rails 2.2.x + GlassFish v3
- #71: JRuby and GlassFish Integration Test #2: JRuby 1.2.0 RC1 + Rails 2.2.x + GlassFish v3 Prelude
- #70: JRuby and GlassFish Integration Test# 1: JRuby 1.2.0 RC1 + Rails 2.2.x + GlassFish Gem
- #69: GlassFish High Availability/Clustering using Sun Web Server + Load Balancer Plugin on Windows Vista
- #68: Installing Zones in Open Solaris 2008/11 on Virtual Box
- #67: How to front-end a GlassFish Cluster with Apache + mod_jk on Mac OSX Leopard ?
- #66: GlassFish Eclipse Plugin 1.0.16 - Install v3 Prelude from the IDE
- #65: Windows 7 Beta 1 Build 7000 on Virtual Box: NetBeans + Rails + GlassFish + MySQL
- #64: OpenSolaris 2008/11 using Virtual Box
- #63: jmx4r gem - How to manage/monitor your Rails/Merb applications on JRuby/GlassFish ?
- #62: How to remotely manage/monitor your Rails/Merb applications on JRuby/GlassFish using JMX API ?
- #61: How to locally manage/monitor your Rails/Merb applications on JRuby/GlassFish using JMX ?
- #60: Configure MySQL 6.0.x-alpha to NetBeans 6.5
- #59: How to add Twitter feeds to blogs.sun.com ? + Other Twitter Tools
- #58: Jersey and GlassFish - how to process POST requests ?
- #57: Jersey Client API - simple and easy to use
- #56: Simple RESTful Web service using Jersey and Embeddable GlassFish - Text and JSON output
- #55: How to build GlassFish v3 Gem ?
- #54: Java Server Faces with Eclipse IDE
- #53: Scaffold in Merb using JRuby/GlassFish
- #52: Getting Started with Merb using GlassFish Gem
- #51: Embedding Google Maps in Java Server Faces using GMaps4JSF
- #50: Mojarra 2.0 EDR2 is now available - Try them with GlassFish v3 and NetBeans 6.5
- #49: Converting a JSF 1.2 application to JSF 2.0 - @ManagedBean
- #48: Converting a JSF 1.2 application to JSF 2.0 - Facelets and Ajax
- #47: Getting Started with Mojarra 2.0 nightly on GlassFish v2
- #46: Facelets with Java Server Faces 1.2
- #45: Ajaxifying Java Server Faces using JSF Extensions
- #44: JDBC Connection Pooling for Rails on GlassFish v3
- #43: GlassFish v3 Build Flavors
- #42: Hello JavaServer Faces World with NetBeans and GlassFish
- #41: How I created transparent logo of GlassFish using Gimp ?
- #40: jQuery Autcomplete widget with MySQL, GlassFish, NetBeans
- #39: Prototype/Script.aculo.us Autcomplete widget with MySQL, GlassFish, NetBeans
- #38: Creating a MySQL Persistence Unit using NetBeans IDE
- #37: SQLite3 with Ruby-on-Rails on GlassFish Gem
- #36: Writing First Test for a Rails Application
- #35: Rails Database Connection on Solaris
- #34: Using Felix Shell with GlassFish
- #33: Building GlassFish v3 Workspace
- #32: Rails Deployment on GlassFish v3 from NetBeans IDE
- #31: CRUD Application using Grails - Hosted on GlassFish and MySQL
- #30: CRUD Application using Grails - Hosted on Jetty and HSQLDB
- #29: Enabling "Available Plugins" tab in NetBeans IDE
- #28: Getting Started with Rails 2.0 Scaffold
- #27: Configurable Multiple Ruby Platforms in NetBeans 6.1 M1
- #26: Overriding Database Defaults in Rails 2.0.2
- #25: Rails application with PostgreSQL database using NetBeans
- #24: Getting Started with Rails 2.0.x in JRuby 1.0.3 and JRuby 1.1RC1
- #23: JavaFX Client invoking a Metro endpoint
- #22: Java SE client for a Metro endpoint
- #21: Metro 1.1 with GlassFish v2 UR1 and NetBeans 6
- #20: How to create a new jMaki widget ?
- #19: How to Add Metro Quality-of-Service to Contract-First Endpoint ?
- #18: How to Build The GlassFish v3 Gem for JRuby ?
- #17: Backing Up your Blog Posts on Roller
- #16: Optimizing Metro Stubs by locally packaging the WSDL
- #15: Delete/Update Row from Database using jMaki Data Table
- #14: How to generate JRuby-on-Rails Controller on Windows (#9893)
- #13: Setup Mongrel for JRuby-on-Rails applications on Windows
- #12: Invoking a Java EE 5 Web service endpoint from JRuby
- #11: Setup Mongrel cluster for JRuby-on-Rails applications on Unix
- #10: Consuming JSON and XML representations generated by a Jersey endpoint in a jMaki Table widget
- #9: Using JDBC connection pool/JNDI name from GlassFish in Rails Application
- #8: Generating JSON using JAXB annotations in Jersey
- #7: Switch between JRuby and CRuby interpreter in NetBeans 6
- #6: Difference between Ruby Gem and Rails Plugin
- #5: Loading data from beans in jMaki widgets
- #4: How to convert a Session EJB to a Web service ?
- #3: Using JavaDB with JRuby on Rails
- #2: Change the endpoint address on a pre-generated Web services Stub
- #1: SOAP Messaging Logging in Metro
Just for fun, here is another tag cloud:
You can access all the tips here. And keep those suggestions coming!
Technorati: totd glassfish netbeans jpa jsf jaxws jersey mysql rails osgi eclipse
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:10:"Arun Gupta";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:28;a:6:{s:4:"data";s:68:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:20:"Logs memory pressure";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:20:"http://dom.as/?p=818";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:46:"http://dom.as/2010/11/18/logs-memory-pressure/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:5209:"Warning, this may be kernel version specific, albeit this kernel is used by many database systems
Lately I’ve been working on getting more memory used by InnoDB buffer pool – besides obvious things like InnoDB memory tax there were seemingly external factors that were pushing out MySQL into swap (even with swappiness=0). We were working a lot on getting low hanging fruits like scripts that use too much memory, but they seem to be all somewhat gone, but MySQL has way too much memory pressure from outside.
I grabbed my uncache utility to assist with the investigation and started uncaching various bits on two systems, one that had larger buffer pool (60G), which was already being sent to swap, and a conservatively allocated (55G) machine, both 72G boxes. Initial finds were somewhat surprising – apparently on both machines most of external-to-mysqld memory was conserved by two sets of items:
binary logs – write once, read only tail (sometimes, if MySQL I/O cache cannot satisfy) – we saw nearly 10G consumed by binlogs on conservatively allocated machines
transaction logs – write many, read never (by MySQL), buffered I/O – full set of transaction logs was found in memory
It was remarkably easy to get rid of binlogs from cache, both by calling out ‘uncache’ from scripts, or using this tiny Python class:
libc = ctypes.CDLL("libc.so.6")
class cachedfile (file):
FADV_DONTNEED = 4
def uncache(self):
libc.posix_fadvise(self.fileno(), 0, 0, self.FADV_DONTNEED)
As it was major memory stress source, it was somewhat a no brainer that binlogs have to be removed from cache – something that can be serially re-read is taking space away from a buffer pool which avoids random reads. It may make sense to call posix_fadvise() right after writes to them, even.
Transaction logs, on the other hand, are entirely different beast. From MySQL perspective they should be uncached immediately, as nobody ever ever reads them (crash recovery aside, but re-reading then is relatively cheap, as no writes or random reads are done during log read phase). Unfortunately, the problem lies way below MySQL, and thanks to PeterZ for reminding me (we had a small chat about this at Jeremy’s Silicon Valley MySQL Meetup).
MySQL transaction records are stored in multiple log groups per transaction, then written out as per-log-group writes (each is in multiple of 512 bytes), followed by fsync(). This allows FS to do transaction log write as single I/O operation. This also means that it will be doing partial page writes to buffered files – overwriting existing data in part of the page, so it has to be read from storage.
So, if all transaction log pages are removed from cache, quite some of them will have to be read back in (depending on sizes of transactions, probably all of them in some cases). Oddly enough, when I tried to hit the edge case, single thread transactions-per-second remained same, but I saw consistent read I/O traffic on disks. So, this would probably work on systems, that have spare I/O (e.g. flash based ones).
Of course, as writes are already in multiples of 512 (and appears that memory got allocated just fine), I could try out direct I/O – it should avoid page read-in problem and not cause any memory pressure by itself. In this case switching InnoDB to use O_DIRECT was a bit dirtier – one needs to edit source code and rebuild the server, restart, etc, or…
# lsof ib_logfile*
# gdb -p $(pidof mysqld)
(gdb) call os_file_set_nocache(9, "test", "test")
(gdb) call os_file_set_nocache(10, "test", "test")
I did not remove fsync() call, but as it is somewhat noop on O_DIRECT files, I left it there, probably it would change benchmark results, but not much.
Some observations:
O_DIRECT was ~10% faster at best case scenario – lots of tiny transactions in single thread
If group commit is used (without binlogs), InnoDB can have way more transactions with multiple threads using buffered I/O, as it does multiple writes per fsync
Enabling sync_binlog makes the difference not that big – even with many parallel writes direct writes are 10-20% slower than buffered ones
Same for innodb_flush_log_on_trx_commit0 – multiple writes per fsync are much more efficient with buffered I/O
One would need to do log group merge to have more efficient O_DIRECT for larger transactions
O_DIRECT does not have theoretical disadvantage, current deficiencies are just implementation oriented at buffered I/O – and can be resolved by (in same areas – extensive) engineering
YMMV. In certain cases it definitely makes sense even right now, in some other – not so much
So, the outcome here depends on many variables – with flash read-on-write is not as expensive, especially if read-ahead works. With disks one has to see what is better use for the memory – using it for buffer pool reduces amount of data reads, but causes log reads. And of course, O_DIRECT wins in the long run :-)
With this data moved away from cache and InnoDB memory tax reduced one could switch from using 75 % of memory to 90% or even 95% for InnoDB buffer pools. Yay?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 12:59:33 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:8:"facebook";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"directio";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:2:"io";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:6:"memory";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5862:"Warning, this may be kernel version specific, albeit this kernel is used by many database systems
Lately I’ve been working on getting more memory used by InnoDB buffer pool – besides obvious things like InnoDB memory tax there were seemingly external factors that were pushing out MySQL into swap (even with swappiness=0). We were working a lot on getting low hanging fruits like scripts that use too much memory, but they seem to be all somewhat gone, but MySQL has way too much memory pressure from outside.
I grabbed my uncache utility to assist with the investigation and started uncaching various bits on two systems, one that had larger buffer pool (60G), which was already being sent to swap, and a conservatively allocated (55G) machine, both 72G boxes. Initial finds were somewhat surprising – apparently on both machines most of external-to-mysqld memory was conserved by two sets of items:
- binary logs – write once, read only tail (sometimes, if MySQL I/O cache cannot satisfy) – we saw nearly 10G consumed by binlogs on conservatively allocated machines
- transaction logs – write many, read never (by MySQL), buffered I/O – full set of transaction logs was found in memory
It was remarkably easy to get rid of binlogs from cache, both by calling out ‘uncache’ from scripts, or using this tiny Python class:
libc = ctypes.CDLL("libc.so.6")
class cachedfile (file):
FADV_DONTNEED = 4
def uncache(self):
libc.posix_fadvise(self.fileno(), 0, 0, self.FADV_DONTNEED)
As it was major memory stress source, it was somewhat a no brainer that binlogs have to be removed from cache – something that can be serially re-read is taking space away from a buffer pool which avoids random reads. It may make sense to call posix_fadvise() right after writes to them, even.
Transaction logs, on the other hand, are entirely different beast. From MySQL perspective they should be uncached immediately, as nobody ever ever reads them (crash recovery aside, but re-reading then is relatively cheap, as no writes or random reads are done during log read phase). Unfortunately, the problem lies way below MySQL, and thanks to PeterZ for reminding me (we had a small chat about this at Jeremy’s Silicon Valley MySQL Meetup).
MySQL transaction records are stored in multiple log groups per transaction, then written out as per-log-group writes (each is in multiple of 512 bytes), followed by fsync(). This allows FS to do transaction log write as single I/O operation. This also means that it will be doing partial page writes to buffered files – overwriting existing data in part of the page, so it has to be read from storage.
So, if all transaction log pages are removed from cache, quite some of them will have to be read back in (depending on sizes of transactions, probably all of them in some cases). Oddly enough, when I tried to hit the edge case, single thread transactions-per-second remained same, but I saw consistent read I/O traffic on disks. So, this would probably work on systems, that have spare I/O (e.g. flash based ones).
Of course, as writes are already in multiples of 512 (and appears that memory got allocated just fine), I could try out direct I/O – it should avoid page read-in problem and not cause any memory pressure by itself. In this case switching InnoDB to use O_DIRECT was a bit dirtier – one needs to edit source code and rebuild the server, restart, etc, or…
# lsof ib_logfile*
# gdb -p $(pidof mysqld)
(gdb) call os_file_set_nocache(9, "test", "test")
(gdb) call os_file_set_nocache(10, "test", "test")
I did not remove fsync() call, but as it is somewhat noop on O_DIRECT files, I left it there, probably it would change benchmark results, but not much.
Some observations:
- O_DIRECT was ~10% faster at best case scenario – lots of tiny transactions in single thread
- If group commit is used (without binlogs), InnoDB can have way more transactions with multiple threads using buffered I/O, as it does multiple writes per fsync
- Enabling sync_binlog makes the difference not that big – even with many parallel writes direct writes are 10-20% slower than buffered ones
- Same for innodb_flush_log_on_trx_commit<>0 – multiple writes per fsync are much more efficient with buffered I/O
- One would need to do log group merge to have more efficient O_DIRECT for larger transactions
- O_DIRECT does not have theoretical disadvantage, current deficiencies are just implementation oriented at buffered I/O – and can be resolved by (in same areas – extensive) engineering
- YMMV. In certain cases it definitely makes sense even right now, in some other – not so much
So, the outcome here depends on many variables – with flash read-on-write is not as expensive, especially if read-ahead works. With disks one has to see what is better use for the memory – using it for buffer pool reduces amount of data reads, but causes log reads. And of course, O_DIRECT wins in the long run :-)
With this data moved away from cache and InnoDB memory tax reduced one could switch from using 75 % of memory to 90% or even 95% for InnoDB buffer pools. Yay?
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Domas Mituzas";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:29;a:6:{s:4:"data";s:43:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:39:"Which SQL Mode Should I Use with MySQL?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-20434908.post-5123927606292217400";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:76:"http://blog.shlomoid.com/2010/11/which-sql-mode-should-i-use-with-mysql.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1184:"Dealing with sql_mode can be a tricky business, and many people have blogged about it before (here and here). My intent is not to explain the details of what each option means (since it's covered in the manual), but rather to answer this simple and common question:Which SQL mode should I use with MySQL? / What is the recommended SQL mode for MySQL?My general purpose recommendation is this one:sql-mode="TRADITIONAL,NO_ENGINE_SUBSTITUTION,ONLY_FULL_GROUP_BY"This "traditional" setting includes most of what you'd want your database to behave like, namely:STRICT_TRANS_TABLES, STRICT_ALL_TABLES, NO_ZERO_IN_DATE,NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USERThis has a nice benefit of returning an error instead of a warning on truncating a varchar column, not let you inserts completely broken date values, and give out division by zero errors. That's pretty traditional as I see it.The other two settings make sure you don't create InnoDB tables as MyISAM by mistake, and that you include all the columns in the GROUP BY clause, like in other databases.That's my choice for a reasonable all-around setting - for different applications your might need different ones.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 11:34:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1870:"Dealing with sql_mode can be a tricky business, and many people have blogged about it before (here and here). My intent is not to explain the details of what each option means (since it's covered in the manual), but rather to answer this simple and common question:
Which SQL mode should I use with MySQL? / What is the recommended SQL mode for MySQL?
My general purpose recommendation is this one:
sql-mode="TRADITIONAL,NO_ENGINE_SUBSTITUTION,ONLY_FULL_GROUP_BY"
This "traditional" setting includes most of what you'd want your database to behave like, namely:
STRICT_TRANS_TABLES, STRICT_ALL_TABLES, NO_ZERO_IN_DATE,NO_ZERO_DATE, ERROR_FOR_DIVISION_BY_ZERO, NO_AUTO_CREATE_USER
This has a nice benefit of returning an error instead of a warning on truncating a varchar column, not let you inserts completely broken date values, and give out division by zero errors. That's pretty traditional as I see it.
The other two settings make sure you don't create InnoDB tables as MyISAM by mistake, and that you include all the columns in the GROUP BY clause, like in other databases.
That's my choice for a reasonable all-around setting - for different applications your might need different ones.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Shlomo Priymak";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:30;a:6:{s:4:"data";s:38:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:43:"What information do you want in the binlog?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:70:"tag:blogger.com,1999:blog-2113389954677193299.post-1839465004177979157";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:84:"http://larsthalmann.blogspot.com/2010/11/what-information-do-you-want-in-binlog.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:458:"The MySQL task WL#4033 Informational Events includes adding information about the original query for row-based replication. This will make it easier to debug failures and user mistakes. It also makes it easier for us to add other information to the binary log. Is there some information missing that you think we should add to the binary log? Please feel free to comment on this blog entry or send me an email at firstname.lastname@oracle.com.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 11:32:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:899:"The MySQL task WL#4033 Informational Events includes adding information about the original query for row-based replication. This will make it easier to debug failures and user mistakes. It also makes it easier for us to add other information to the binary log. Is there some information missing that you think we should add to the binary log? Please feel free to comment on this blog entry or send me an email at firstname.lastname@oracle.com.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Lars Thalmann";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:31;a:6:{s:4:"data";s:78:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:44:"Transforming Telecommunications with Web 2.0";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:44:"tag:blogs.oracle.com,2010:/mysql//3342.23755";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:86:"http://blogs.oracle.com/mysql/2010/11/transforming_telecommunications_with_web_20.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2579:"The convergence of telecommunications onto IP (Internet Protocol) based networks is revolutionizing today's communications industry. Communications Service Providers (CSPs) are challenged to deliver compelling new personalized services with greater agility and lower costs than ever before.Whether they are looking to deploy new Web/Telco 2.0 applications to mobile Internet users or consolidating subscriber data within the network to support greater service personalization and targeted communications, the database plays a key role in enabling new services.We believe MySQL is pretty much unique in the industry by providing the insight and experience to marry the innovation and speed of the Web with the proven capabilities of the carrier network. MySQL is deployed in 9 of the top 10 most trafficked sites on the web [1] including Google, Facebook and YouTube. MySQL is also extensively deployed in the network domain, powering real-time subscriber and service delivery solutions for the likes of Alcatel-Lucent, BT Plusnet and Telenor where carrier-grade availability and performance are critical.And the fact that MySQL can do this with open-source technology means that you can reduce costs while you improve services and deliver them to market faster.This combination of web-scale performance and carrier-grade availability is what lead Nokia to select MySQL to power its data platform for large-scale, mission-critical Web-based services accessed via a variety of smartphone applications. Yekesa Kosuru, Distinguished Architect at Nokia won the Oracle award for MySQL Developer of the Year 2010 as a result of this project You can learn more about how MySQL is used in telecommunications from our upcoming webinar scheduled on December 15th. Don't worry if you can't attend live - by registering you will be automatically notified when the on-demand event replay is availableTo understand the adoption and impact of open source software within the telecommunications industry, IDC has also published a whitepaper which we've made freely available by registering for the download here As Service Providers reposition themselves to become the central enabler of social and business communications services, so the value delivered by IT solutions in general and data management technology in particular becomes increasingly critical. As the examples above demonstrate, you weren't go far wrong in looking at MySQL as a data platform for new service innovation. You might even win a nice award as well ![1] http://www.alexa.com/topsites";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 10:53:24 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:13:"MySQL Cluster";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"MySQL Server";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:8:"Mat Keep";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:13:"MySQL Cluster";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:18:"Telecommunications";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:8:"Telecoms";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3475:"The convergence of telecommunications onto IP (Internet Protocol) based networks is revolutionizing today's communications industry. Communications Service Providers (CSPs) are challenged to deliver compelling new personalized services with greater agility and lower costs than ever before.
Whether they are looking to deploy new Web/Telco 2.0 applications to mobile Internet users or consolidating subscriber data within the network to support greater service personalization and targeted communications, the database plays a key role in enabling new services.
We believe MySQL is pretty much unique in the industry by providing the insight and experience to marry the innovation and speed of the Web with the proven capabilities of the carrier network. MySQL is deployed in 9 of the top 10 most trafficked sites on the web [1] including Google, Facebook and YouTube. MySQL is also extensively deployed in the network domain, powering real-time subscriber and service delivery solutions for the likes of Alcatel-Lucent, BT Plusnet and Telenor where carrier-grade availability and performance are critical.
And the fact that MySQL can do this with open-source technology means that you can reduce costs while you improve services and deliver them to market faster.
This combination of web-scale performance and carrier-grade availability is what lead Nokia to select MySQL to power its data platform for large-scale, mission-critical Web-based services accessed via a variety of smartphone applications. Yekesa Kosuru, Distinguished Architect at Nokia won the Oracle award for MySQL Developer of the Year 2010 as a result of this project
You can learn more about how MySQL is used in telecommunications from our upcoming webinar scheduled on December 15th. Don't worry if you can't attend live - by registering you will be automatically notified when the on-demand event replay is available
To understand the adoption and impact of open source software within the telecommunications industry, IDC has also published a whitepaper which we've made freely available by registering for the download here
As Service Providers reposition themselves to become the central enabler of social and business communications services, so the value delivered by IT solutions in general and data management technology in particular becomes increasingly critical. As the examples above demonstrate, you weren't go far wrong in looking at MySQL as a data platform for new service innovation. You might even win a nice award as well !
[1] http://www.alexa.com/topsites
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:18:"Oracle MySQL Group";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:32;a:6:{s:4:"data";s:63:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:66:"10 Steps: MySQL Monitoring through Nagios: Install & Configure";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:41:"http://kedar.nitty-witty.com/blog/?p=1348";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:93:"http://kedar.nitty-witty.com/blog/10-steps-mysql-monitoring-nagios-installation-configuration";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:586:"Nagios is a powerful monitoring system and here we will learn how to monitor MySQL through Nagios. We will be installing Nagios, required plugins and configuring it to monitor MySQL...
Related posts:Installing MySQL On Mac OS X (Darwin Kernel) Recently I happen to install MySQL on Mac OS X...
Steps to Move Copy Rename MySQL Database Moving, copying or renaming database is a very basic activity....
Ideas for select all columns but one mysql stored procedure Assume we’ve a table with 100 rows and we need...
Related posts brought to you by Yet Another Related Posts Plugin.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 10:42:52 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:9:"Technical";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:19:"database monitoring";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:12:"Installation";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:6:"nagios";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1477:"Nagios is a powerful monitoring system and here we will learn how to monitor MySQL through Nagios. We will be installing Nagios, required plugins and configuring it to monitor MySQL...
Related posts:- Installing MySQL On Mac OS X (Darwin Kernel) Recently I happen to install MySQL on Mac OS X...
- Steps to Move Copy Rename MySQL Database Moving, copying or renaming database is a very basic activity....
- Ideas for select all columns but one mysql stored procedure Assume we’ve a table with 100 rows and we need...
Related posts brought to you by Yet Another Related Posts Plugin.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:5:"Kedar";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:33;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:32:"My Presentation at the DOAG 2010";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-24359421.post-5945222126167350956";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:66:"http://pbxt.blogspot.com/2010/11/my-presentation-at-doag-2010.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:787:"Yesterday I presented PBXT: A Transactional Storage Engine for MySQL at the German Oracle User Group Conference (DOAG) in Nuremberg. A number of people asked for the slides, so here is the link.The talk was scheduled to be in English, but since I had a German-only audience I presented in German. There was quite a bit of interest, particularly in the Engine Level replication built into PBXT 2.0.As Ronny observed, this feature can be used effectively for many tasks, including for online backup and maintaining a hot-standby. This all with the addition of a "small" feature:The Master could initially stream the entire database over to the Slave before actual replication begins. This would also make it extremely easy to setup replication.A brilliant idea, but a good 3 months work...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 18 Nov 2010 09:25:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:4:"pbxt";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:4:"doag";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"presentation";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1271:"Yesterday I presented PBXT: A Transactional Storage Engine for MySQL at the German Oracle User Group Conference (DOAG) in Nuremberg. A number of people asked for the slides, so here is the link.
The talk was scheduled to be in English, but since I had a German-only audience I presented in German. There was quite a bit of interest, particularly in the Engine Level replication built into PBXT 2.0.
As Ronny observed, this feature can be used effectively for many tasks, including for online backup and maintaining a hot-standby. This all with the addition of a "small" feature:
The Master could initially stream the entire database over to the Slave before actual replication begins. This would also make it extremely easy to setup replication.
A brilliant idea, but a good 3 months work...
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Paul McCullagh";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:34;a:6:{s:4:"data";s:43:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:41:"Avoiding Fragmentation with Fractal Trees";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:26:"http://tokutek.com/?p=1705";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:69:"http://tokutek.com/2010/11/avoiding-fragmentation-with-fractal-trees/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:4999:"Summary
B-trees suffer from fragmentation. Fragmentation causes headaches — in query performance and space used. Solutions, like dump and reload or OPTIMIZE TABLE are a pain and not always effective. Fractal trees don’t fragment. So if fragmentation is a problem, check out Tokutek
What is fragmentation?
What do I mean when I say “fragmentation”? People complain about two things when they talk about index fragmentation. They either find that the disk spaced used is much larger than the data, or they complain about query performance. In particular, they complain of
range query performance, since point queries aren’t really affected by fragmentation. I’m going to focus on this second symptom, which is due to lack of locality of reference amongst the rows.
B-trees fragment
In the following, I’ll be talking about the limits of B-trees. Not InnoDB in particular, but B-trees in general. InnoDB’s B-tree, and MyISAM’s, and SQL Server’s, and Oracle’s, and …
Consider two types of B-trees: clustered and non-clustered. In the MySQL world, we can think of these and InnoDB and MyISAM. Since I’m talking about locality of reference amongst rows, I’ll treat these separately.
A fragmented non-clustering B-trees is just about as fragmented as you can get. If you insert data in an arbitrary order, the logical order of the rows will be unrelated to the physical placement on disk. Scanning through a chunk of rows will cause the disk head to move around for each row (in the worst case). Non-clustering B-trees are generally not a good choice for range queries.
Clustering B-trees place groups of logically close rows into leaf blocks together. That means that when you fetch a leaf during a range query, you get a bunch of rows that satisfy your WHERE clause, not just one row. Thus, range queries on a clustering B-tree are faster. (Ok, it’s possible to concoct a case where MyISAM does range queries faster than InnoDB, so insert appropriate caveats here, but these are corner cases I’m going
to ignore.)
Inserting into a clustering B-tree is slower than inserting into a non-clustering B-tree. B-trees have a tradeoff between insertion and fragmentation. Important note: this tradeoff isn’t inherent in the indexing problem. It’s a tradeoff that B-trees have.
So can we do even better on range queries — and have even less fragmentation than a clustering B-tree? Yes. If you keep the leaves in order on disk and nicely packed, then range queries go even faster — up to two orders of magnitude faster. In the case of range query performance, fragmentation is synonymous with scattered leaves. And defragmentation is synonymous with getting the leaves back into order.
The most common forms of defragmentation are OPTIMIZE TABLE, though Baron Schwartz
points out that InnoDB defragments the primary key when doing an optimize table. Secondary tables do not get defragmented. In fact, they may end up more fragmented. Alternatively, you can regularly dump and reload. And finally, you could implement a B-tree with defragment-as-you-go heuristics, but these slow down insertions considerably (there’s that pesky tradeoff) and ultimately can’t keep up with a heavily loaded database.
Everything I’ve been saying is about B-trees in general, not just the ones available for MySQL. For example, in a sample Dell/SQL Server reference configuration, a disproportionate amount of hardware is spent on making index rebuilding fast. Why? Because B-trees fragment.
Fractal trees don’t fragment
So all I’ve said so far is true of B-trees. But that doesn’t mean it’s true of all indexing structures. Fractal trees don’t fragment. They can’t fragment. That means the primary table isn’t fragmented, and neither are the secondary indexes. That means they aren’t fragmented if you bulk load or if you trickle load. In short, if fragmentation is an issue for you, Fractal trees are worth looking at as a solution. Fractal trees are available in TokuDB , a MySQL storage engine.
The existence of fractal trees also illustrates that there is no inherent tradeoff between fragmentation and insertion speed. Fractal trees are much faster than B-trees at insertions — up to two orders of magnitude faster — and don’t fragment. So B-trees sit on a tradeoff curve, but not the best possible tradeoff curve.
And finally…
Fragmentation is a complicated matter, and I don’t claim to have said anything exhaustive about it. For example, fragmentation can cause B-tree tables to take up too much disk space, and defragmentation can cause B-tree tables to shrink, although this is a matter of some dispute for InnoDB. Also, it’s not actually that easy to measure fragmentation in a B-tree. My recommendation is to use an index that doesn’t fragment!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 19:10:15 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:8:"TokuView";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5714:"Summary
B-trees suffer from fragmentation. Fragmentation causes headaches — in query performance and space used. Solutions, like dump and reload or OPTIMIZE TABLE are a pain and not always effective. Fractal trees don’t fragment. So if fragmentation is a problem, check out Tokutek
What is fragmentation?
What do I mean when I say “fragmentation”? People complain about two things when they talk about index fragmentation. They either find that the disk spaced used is much larger than the data, or they complain about query performance. In particular, they complain of
range query performance, since point queries aren’t really affected by fragmentation. I’m going to focus on this second symptom, which is due to lack of locality of reference amongst the rows.
B-trees fragment
In the following, I’ll be talking about the limits of B-trees. Not InnoDB in particular, but B-trees in general. InnoDB’s B-tree, and MyISAM’s, and SQL Server’s, and Oracle’s, and …
Consider two types of B-trees: clustered and non-clustered. In the MySQL world, we can think of these and InnoDB and MyISAM. Since I’m talking about locality of reference amongst rows, I’ll treat these separately.
A fragmented non-clustering B-trees is just about as fragmented as you can get. If you insert data in an arbitrary order, the logical order of the rows will be unrelated to the physical placement on disk. Scanning through a chunk of rows will cause the disk head to move around for each row (in the worst case). Non-clustering B-trees are generally not a good choice for range queries.
Clustering B-trees place groups of logically close rows into leaf blocks together. That means that when you fetch a leaf during a range query, you get a bunch of rows that satisfy your WHERE clause, not just one row. Thus, range queries on a clustering B-tree are faster. (Ok, it’s possible to concoct a case where MyISAM does range queries faster than InnoDB, so insert appropriate caveats here, but these are corner cases I’m going
to ignore.)
Inserting into a clustering B-tree is slower than inserting into a non-clustering B-tree. B-trees have a tradeoff between insertion and fragmentation. Important note: this tradeoff isn’t inherent in the indexing problem. It’s a tradeoff that B-trees have.
So can we do even better on range queries — and have even less fragmentation than a clustering B-tree? Yes. If you keep the leaves in order on disk and nicely packed, then range queries go even faster — up to two orders of magnitude faster. In the case of range query performance, fragmentation is synonymous with scattered leaves. And defragmentation is synonymous with getting the leaves back into order.
The most common forms of defragmentation are OPTIMIZE TABLE, though Baron Schwartz
points out that InnoDB defragments the primary key when doing an optimize table. Secondary tables do not get defragmented. In fact, they may end up more fragmented. Alternatively, you can regularly dump and reload. And finally, you could implement a B-tree with defragment-as-you-go heuristics, but these slow down insertions considerably (there’s that pesky tradeoff) and ultimately can’t keep up with a heavily loaded database.
Everything I’ve been saying is about B-trees in general, not just the ones available for MySQL. For example, in a sample Dell/SQL Server reference configuration, a disproportionate amount of hardware is spent on making index rebuilding fast. Why? Because B-trees fragment.
Fractal trees don’t fragment
So all I’ve said so far is true of B-trees. But that doesn’t mean it’s true of all indexing structures. Fractal trees don’t fragment. They can’t fragment. That means the primary table isn’t fragmented, and neither are the secondary indexes. That means they aren’t fragmented if you bulk load or if you trickle load. In short, if fragmentation is an issue for you, Fractal trees are worth looking at as a solution. Fractal trees are available in TokuDB , a MySQL storage engine.
The existence of fractal trees also illustrates that there is no inherent tradeoff between fragmentation and insertion speed. Fractal trees are much faster than B-trees at insertions — up to two orders of magnitude faster — and don’t fragment. So B-trees sit on a tradeoff curve, but not the best possible tradeoff curve.
And finally…
Fragmentation is a complicated matter, and I don’t claim to have said anything exhaustive about it. For example, fragmentation can cause B-tree tables to take up too much disk space, and defragmentation can cause B-tree tables to shrink, although this is a matter of some dispute for InnoDB. Also, it’s not actually that easy to measure fragmentation in a B-tree. My recommendation is to use an index that doesn’t fragment!
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:20:"Martin Farach-Colton";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:35;a:6:{s:4:"data";s:68:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:24:"Developer Week in Review";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:37:"tag:radar.oreilly.com,2010://57.43378";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:96:"http://feedproxy.google.com/~r/oreilly/radar/atom/~3/H9OcqVhUfxY/developer-week-in-review-6.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:4028:"Here's what's new for the trendy developer this week:
Java's future on Apple: Slightly less in doubt
Last week, it looked like Apple was all "You're not welcome here, Java." In the changeable world that is Jobsland, this week Apple was offering to marry the language, reiterating their support for Java in OS X, and indicating that they would be supplying code and resources to the OpenJDK project.
As I've noted before, this makes sense for Apple, because it gets them out of the JVM business, and makes Oracle the one-stop shopping solution for all your JDK and JRE needs. It also means that the Mac can be added as a regression-tested target for a new version of Java, hopefully avoiding the kind of Java versioning snafus that rendered IBM's SPSS (or is it PAWS this week?) statistics package broken for the last month or so.
Apple makes nice with Java and lets the Google Voice app hit the iPhone in the same week. Could peace in the Middle East be next?
Editorial: NPEs must die
Staying on a Java theme, probably the most common error messages I see on websites (after the all-too-common IIS MSQL database) are Java Null Pointer Exceptions (NPE). If you're not a Java person, an NPE is what you get when you try to call a method on an object, but the variable holding the object contains null, rather than a real object.
NPEs are to Java what bad pointer references are to C. The difference is that, unlike C, an NPE in Java is usually a non-fatal event. In a web server container like Tomcat, an uncaught NPE ends up as status code 500 HTTP backtrace spam.
NPEs are a direct result of not checking the validity of arguments and data before operating on it. Too many Java programmers are lazy, and assume because an NPE doesn't crash things, they can just let it trickle up the stack until someone catches it.
This is just plain bad coding. An NPE tells you almost nothing about what really went wrong. Instead, Java developers should check objects before invoking methods on them, and throw a more specific and meaningful runtime exception, such as an IllegalArgumentException, with details about what value was incorrect.
The same holds true for other languages, of course. It's always a best practice to verify values external to the current context. Java developers just seem to be especially bad about it. So remember folks, trust but verify!
And speaking of MSQL
This must be my week for segues, because speaking of MSQL, Microsoft released a CTP version of SQL Server 2011 (Denali, they call it). CTP is Community Technology Preview, which is kinda Microsoftease for "beta."
Love it or hate it, MSQL powers a lot of the web, and a new turn of SQL Server is a big deal for that part of the world that runs on 96-octane .NET fuel. Database technology is pretty much a two-horse game these days, now that Oracle owns the former third horse (MySQL.) A new SQL Server will hopefully raise the bar enough to get some new and interesting things out of Oracle's two database projects.
Another day on the JavaScript racetrack
Probably the most commonly programmed platform these days is JavaScript on the browser. As a result, browser vendors take their JavaScript performance numbers very seriously. For a while, Chrome has been the target to beat. Ok, technically, Opera has been posting better numbers lately, but Opera has never managed to build significant market share. Let's just say that Chrome has had the best numbers among the big 4 (IE, Firefox, Chrome, Safari).
Mozilla evidently decided not to take things lying down. The latest version of the Firefox 4 beta has turned in speed numbers the Road Runner would find respectable. How long will it take Google to respond? Or, will the new IE beta come from behind to snatch the crown of JavaScript performance? Personally, I'd prefer that Google fix Chrome so Flash doesn't break on the Mac if you look at it funny ...
That's it for this week. Suggestions are always welcome, so please send tips or news here.
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 15:30:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:11:"Programming";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:12:"developerwir";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:4:"Java";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:10:"JavaScript";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:11:"programming";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:3:"SQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5801:"Here's what's new for the trendy developer this week:
Java's future on Apple: Slightly less in doubt
Last week, it looked like Apple was all "You're not welcome here, Java." In the changeable world that is Jobsland, this week Apple was offering to marry the language, reiterating their support for Java in OS X, and indicating that they would be supplying code and resources to the OpenJDK project.
As I've noted before, this makes sense for Apple, because it gets them out of the JVM business, and makes Oracle the one-stop shopping solution for all your JDK and JRE needs. It also means that the Mac can be added as a regression-tested target for a new version of Java, hopefully avoiding the kind of Java versioning snafus that rendered IBM's SPSS (or is it PAWS this week?) statistics package broken for the last month or so.
Apple makes nice with Java and lets the Google Voice app hit the iPhone in the same week. Could peace in the Middle East be next?
Editorial: NPEs must die
Staying on a Java theme, probably the most common error messages I see on websites (after the all-too-common IIS MSQL database) are Java Null Pointer Exceptions (NPE). If you're not a Java person, an NPE is what you get when you try to call a method on an object, but the variable holding the object contains null, rather than a real object.
NPEs are to Java what bad pointer references are to C. The difference is that, unlike C, an NPE in Java is usually a non-fatal event. In a web server container like Tomcat, an uncaught NPE ends up as status code 500 HTTP backtrace spam.
NPEs are a direct result of not checking the validity of arguments and data before operating on it. Too many Java programmers are lazy, and assume because an NPE doesn't crash things, they can just let it trickle up the stack until someone catches it.
This is just plain bad coding. An NPE tells you almost nothing about what really went wrong. Instead, Java developers should check objects before invoking methods on them, and throw a more specific and meaningful runtime exception, such as an IllegalArgumentException, with details about what value was incorrect.
The same holds true for other languages, of course. It's always a best practice to verify values external to the current context. Java developers just seem to be especially bad about it. So remember folks, trust but verify!
And speaking of MSQL
This must be my week for segues, because speaking of MSQL, Microsoft released a CTP version of SQL Server 2011 (Denali, they call it). CTP is Community Technology Preview, which is kinda Microsoftease for "beta."
Love it or hate it, MSQL powers a lot of the web, and a new turn of SQL Server is a big deal for that part of the world that runs on 96-octane .NET fuel. Database technology is pretty much a two-horse game these days, now that Oracle owns the former third horse (MySQL.) A new SQL Server will hopefully raise the bar enough to get some new and interesting things out of Oracle's two database projects.
Another day on the JavaScript racetrack
Probably the most commonly programmed platform these days is JavaScript on the browser. As a result, browser vendors take their JavaScript performance numbers very seriously. For a while, Chrome has been the target to beat. Ok, technically, Opera has been posting better numbers lately, but Opera has never managed to build significant market share. Let's just say that Chrome has had the best numbers among the big 4 (IE, Firefox, Chrome, Safari).
Mozilla evidently decided not to take things lying down. The latest version of the Firefox 4 beta has turned in speed numbers the Road Runner would find respectable. How long will it take Google to respond? Or, will the new IE beta come from behind to snatch the crown of JavaScript performance? Personally, I'd prefer that Google fix Chrome so Flash doesn't break on the Mac if you look at it funny ...
That's it for this week. Suggestions are always welcome, so please send tips or news here.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Tim O'Reilly";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:36;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:62:"Cloud Challenges for Databases Series: Distributed Environment";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:30:"http://blog.xeround.com/?p=201";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:97:"http://blog.xeround.com/2010/11/17/cloud-challenges-for-databases-series-distributed-environment/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3632:"When it comes to cloud computing, people want to command nearly endless resources while still being able to deploy, run and consume a distributed solution anywhere and anytime. The reality, however, is not that simple.
Cloud computing relies on the interconnection of resources. Network latency usually plays a critical role in the application flow from the front end, through the middle tier to the database and all the way back again.
Setting mobile access aside for the sake of this discussion, the performance one would get would depend on the proximity to the cloud entry point (datacenter). For example, if your SaaS application is deployed in the Amazon EC2 West Availability Zone, users on the East Coast would not get the same level of performance as those on the West Coast. That’s reality at work – no bending of physical laws.
One way to work around this issue is to deploy the application in several geographical locations (in this case East and West Coasts) so that the majority of users closer to each location will enjoy better performance.
Pursuing this solution assumes a couple key architectural and configuration considerations:
The application is stateless and therefore agnostic of the servers that are actually active in the flow.
The database can be replicated to several locations.
The database’s replicas can operate in multi-master mode.
Running a database with multiple replicas is usually done using a single master that ships updates to several passive copies. The passive copies can be used to offload read operations, but not writes.
Regardless of the exact replication method, failing over (and ultimately back again) from master to a stand-by passive node takes time and can be painful. And if you need to have your database support heavy-duty online transaction processing that entails both reads and writes simultaneously, in several locations — that’s where it gets interesting.
Xeround addressed this specific challenge early on when we designed our solution as a subscriber database management system for telecoms. The very nature of the solution had to be distributed, serving roaming users in multiple locations — a core GSM function. Users would use and access the nearest database.
After moving to the cloud, we refreshed this core capability and applied the same principles to multiple copies of the same database that span multiple locations. In fact, since our core technology allows us to effectively and transparently distribute the database across multiple nodes, it is not really an issue if the same data is spread across multiple nodes or locations – the basic logic remains the same.
At this point, we’re often asked about the “CAP Theorem.” While it’s true that we can’t defy the laws of physics, we can certainly track, manage and synchronize multiple copies of the database and clear conflicts as and where applicable.
To do this, we keep a transaction log constantly updated in the same way we write transactions to multiple replicas. We organically provide this service out of the gate to ensure high availability. We’ve dubbed this concept “Active Global”, because groups of geographically-dispersed clusters of MySQL are accessed and managed locally while they’re synchronized with a top-level, global logic layer. Active Global ensures for near real-time “clearance” anytime, anywhere.
Read more about the challenges databases face when running on a cloud environment on our Cloud Challenges for Databases Series.
Filed under: Cloud Challenges for Databases Series, MySQL Cloud Database Tagged: distribution ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 14:00:30 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:37:"Cloud Challenges for Databases Series";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:20:"MySQL Cloud Database";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"distribution";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:6542:"
When it comes to cloud computing, people want to command nearly endless resources while still being able to deploy, run and consume a distributed solution anywhere and anytime. The reality, however, is not that simple.
Cloud computing relies on the interconnection of resources. Network latency usually plays a critical role in the application flow from the front end, through the middle tier to the database and all the way back again.
Setting mobile access aside for the sake of this discussion, the performance one would get would depend on the proximity to the cloud entry point (datacenter). For example, if your SaaS application is deployed in the Amazon EC2 West Availability Zone, users on the East Coast would not get the same level of performance as those on the West Coast. That’s reality at work – no bending of physical laws.
One way to work around this issue is to deploy the application in several geographical locations (in this case East and West Coasts) so that the majority of users closer to each location will enjoy better performance.
Pursuing this solution assumes a couple key architectural and configuration considerations:
- The application is stateless and therefore agnostic of the servers that are actually active in the flow.
- The database can be replicated to several locations.
- The database’s replicas can operate in multi-master mode.
Running a database with multiple replicas is usually done using a single master that ships updates to several passive copies. The passive copies can be used to offload read operations, but not writes.
Regardless of the exact replication method, failing over (and ultimately back again) from master to a stand-by passive node takes time and can be painful. And if you need to have your database support heavy-duty online transaction processing that entails both reads and writes simultaneously, in several locations — that’s where it gets interesting.
Xeround addressed this specific challenge early on when we designed our solution as a subscriber database management system for telecoms. The very nature of the solution had to be distributed, serving roaming users in multiple locations — a core GSM function. Users would use and access the nearest database.
After moving to the cloud, we refreshed this core capability and applied the same principles to multiple copies of the same database that span multiple locations. In fact, since our core technology allows us to effectively and transparently distribute the database across multiple nodes, it is not really an issue if the same data is spread across multiple nodes or locations – the basic logic remains the same.
At this point, we’re often asked about the “CAP Theorem.” While it’s true that we can’t defy the laws of physics, we can certainly track, manage and synchronize multiple copies of the database and clear conflicts as and where applicable.
To do this, we keep a transaction log constantly updated in the same way we write transactions to multiple replicas. We organically provide this service out of the gate to ensure high availability. We’ve dubbed this concept “Active Global”, because groups of geographically-dispersed clusters of MySQL are accessed and managed locally while they’re synchronized with a top-level, global logic layer. Active Global ensures for near real-time “clearance” anytime, anywhere.
Read more about the challenges databases face when running on a cloud environment on our Cloud Challenges for Databases Series.
Filed under: Cloud Challenges for Databases Series, MySQL Cloud Database Tagged: distribution 
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Avigail Ofer";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:37;a:6:{s:4:"data";s:58:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:37:"MySQL Cluster 7.1.9 binaries released";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.clusterdb.com/?p=1441";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:77:"http://www.clusterdb.com/mysql-cluster/mysql-cluster-7-1-9-binaries-released/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:449:"The binary version for MySQL Cluster 7.1.9 has now been made available at http://www.mysql.com/downloads/cluster/
A description of all of the changes (fixes) that have gone into MySQL Cluster 7.1.9 (compared to 7.1.8) can be found in the official MySQL Cluster documentation. In addition, there is a great BLOG posting from Johan Anderson explaining how to use the new table added to ndbinfo to tune DiskPageBufferMemory when storing tables on disk.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 13:14:11 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:13:"MySQL Cluster";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:17:"MySQL Cluster 7.1";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:17:"MySQL Cluster CGE";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1274:"
The binary version for MySQL Cluster 7.1.9 has now been made available at http://www.mysql.com/downloads/cluster/
A description of all of the changes (fixes) that have gone into MySQL Cluster 7.1.9 (compared to 7.1.8) can be found in the official MySQL Cluster documentation. In addition, there is a great BLOG posting from Johan Anderson explaining how to use the new table added to ndbinfo to tune DiskPageBufferMemory when storing tables on disk.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Andrew Morgan";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:38;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:38:"MySQL Cluster 6.3.39 binaries released";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.clusterdb.com/?p=1438";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:78:"http://www.clusterdb.com/mysql-cluster/mysql-cluster-6-3-39-binaries-released/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:278:"The binary version for MySQL Cluster 6.3.39 has now been made available at http://www.mysql.com/downloads/cluster/
A description of all of the changes (fixes) that have gone into MySQL Cluster 6.3.39 (compared to 6.3.38) can be found in the official MySQL Cluster documentation.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 13:09:44 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:13:"MySQL Cluster";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:17:"MySQL Cluster CGE";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:989:"The binary version for MySQL Cluster 6.3.39 has now been made available at http://www.mysql.com/downloads/cluster/
A description of all of the changes (fixes) that have gone into MySQL Cluster 6.3.39 (compared to 6.3.38) can be found in the official MySQL Cluster documentation.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Andrew Morgan";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:39;a:6:{s:4:"data";s:73:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:47:"The Casual MySQL DBA – Operational Basics";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=3356";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:82:"http://ronaldbradford.com/blog/the-casual-mysql-dba-operational-basics-2010-11-17/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1855:"So your not a MySQL DBA, but you have to perform like one. If you have a production environment that’s running now, what are the first things you do when it’s not running or reported as not running?
Are the MySQL processes running? (i.e. mysqld and mysqld_safe)
Can you connect locally via cli?
What’s in the MySQL error log?
What are current MySQL threads doing? Locked? long running? how many? idle sources?
Can you connect remotely via cli?
Verify free diskspace?
Verify system physical resources?
If this is a slave, is MySQL replication running? Is it up to date?
What is the current MySQL load, e.g. reads/writes/throughput/network/disk etc?
What is the current InnoDB state and load? (based on if your using InnoDB)
After you do this manually more then once you should be scripting these commands to be productive for future analysis and proactive monitoring?
Is a problem obvious? Does the output look different to what a normal environment looks like? (HINT: This list is not just for when there is a problem)
So moving forward?
Is disk/memory/cpu/network bottleneck an issue you can resolve?
Can you improving locking statements (if applicable)?
Can you identify, analyse and tune long running statements?
Do you know how to restart MySQL?
Do you know who to call when you have a non working environment?
When did your backup last run?
Does your last backup work?
In order to support any level of production MySQL environment you need to know the answers to these questions? If you don’t, then this is your homework checklist for MySQL DBA operations 101. There a number of resources where you can find the answers, and this help can be available online, however never assume the timeliness of responses, especially if your expecting if for FREE! Open source software can be free, open source support rarely is.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 11:00:09 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"casual";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:3:"dba";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:7:"locking";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:10:"operations";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2274:"So your not a MySQL DBA, but you have to perform like one. If you have a production environment that’s running now, what are the first things you do when it’s not running or reported as not running?
- Are the MySQL processes running? (i.e. mysqld and mysqld_safe)
- Can you connect locally via cli?
- What’s in the MySQL error log?
- What are current MySQL threads doing? Locked? long running? how many? idle sources?
- Can you connect remotely via cli?
- Verify free diskspace?
- Verify system physical resources?
- If this is a slave, is MySQL replication running? Is it up to date?
- What is the current MySQL load, e.g. reads/writes/throughput/network/disk etc?
- What is the current InnoDB state and load? (based on if your using InnoDB)
After you do this manually more then once you should be scripting these commands to be productive for future analysis and proactive monitoring?
Is a problem obvious? Does the output look different to what a normal environment looks like? (HINT: This list is not just for when there is a problem)
So moving forward?
- Is disk/memory/cpu/network bottleneck an issue you can resolve?
- Can you improving locking statements (if applicable)?
- Can you identify, analyse and tune long running statements?
- Do you know how to restart MySQL?
- Do you know who to call when you have a non working environment?
- When did your backup last run?
- Does your last backup work?
In order to support any level of production MySQL environment you need to know the answers to these questions? If you don’t, then this is your homework checklist for MySQL DBA operations 101. There a number of resources where you can find the answers, and this help can be available online, however never assume the timeliness of responses, especially if your expecting if for FREE! Open source software can be free, open source support rarely is.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:40;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:69:"MySQL (including Cluster) at the 2010 UK Oracle User Group Conference";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.clusterdb.com/?p=1422";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:107:"http://www.clusterdb.com/mysql-cluster/mysql-including-cluster-at-the-2010-uk-oracle-user-group-conference/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1907:"
The UK Oracle User Group Conference runs from 29th November through 1st December 2010 – the MySQL content is on Wednesday 1st at the ICC in Birmingham. This is a great chance for MySQL users to find out more on the latest developments and question some of the experts (both from inside and outside Oracle) and for Oracle users who don’t have experience with MySQL to discover what it’s all about.
The day starts with a keynote from Rich Mason (VP responsible for MySQL sales) at 8:45; I’ll be part of a MySQL panel at 9:50 and then Mat Keep and I will be presenting on MySQL Cluster at 14:15. In the Cluster session we’ll will introduce MySQL Cluster and cover the most important features of MySQL Cluster 7.1: ndbinfo; MySQL Cluster Connector/Java and other features that push the limits of MySQL Cluster into new workloads and communities.
ndbinfo presents real-time usage statistics from the MySQL Cluster data nodes as a series of SQL tables, enabling developers and administrators to monitor database performance and optimize their applications.
Designed for Java developers, the MySQL Cluster Connector for Java implements an easy-to-use and high performance native Java interface and OpenJPA plug-in that maps Java classes to tables stored in the MySQL Cluster database.
Finally, we’ll present some real-world case studies and explain some of the new capabilities that are currently in development.
Another session that’s likely to be suited to people interested in MySQL Cluster is a more general presentation on delivering High Availability with MySQL – presented by Mario Beck.
As you’d expect there are many more MySQL sessions to choose from – check the full schedule.
To register for just the MySQL sessions use the MySQL registration page, the full conference registration page is here.
Hope to see as many of you there as possible!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 09:30:02 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:13:"MySQL Cluster";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:17:"MySQL Replication";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3479:"
The UK Oracle User Group Conference runs from 29th November through 1st December 2010 – the MySQL content is on Wednesday 1st at the ICC in Birmingham. This is a great chance for MySQL users to find out more on the latest developments and question some of the experts (both from inside and outside Oracle) and for Oracle users who don’t have experience with MySQL to discover what it’s all about.
The day starts with a keynote from Rich Mason (VP responsible for MySQL sales) at 8:45; I’ll be part of a MySQL panel at 9:50 and then Mat Keep and I will be presenting on MySQL Cluster at 14:15. In the Cluster session we’ll will introduce MySQL Cluster and cover the most important features of MySQL Cluster 7.1: ndbinfo; MySQL Cluster Connector/Java and other features that push the limits of MySQL Cluster into new workloads and communities.
ndbinfo presents real-time usage statistics from the MySQL Cluster data nodes as a series of SQL tables, enabling developers and administrators to monitor database performance and optimize their applications.
Designed for Java developers, the MySQL Cluster Connector for Java implements an easy-to-use and high performance native Java interface and OpenJPA plug-in that maps Java classes to tables stored in the MySQL Cluster database.
Finally, we’ll present some real-world case studies and explain some of the new capabilities that are currently in development.
Another session that’s likely to be suited to people interested in MySQL Cluster is a more general presentation on delivering High Availability with MySQL – presented by Mario Beck.
As you’d expect there are many more MySQL sessions to choose from – check the full schedule.
To register for just the MySQL sessions use the MySQL registration page, the full conference registration page is here.
Hope to see as many of you there as possible!
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Andrew Morgan";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:41;a:6:{s:4:"data";s:63:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:25:"MySQL - My new playground";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:52:"http://www.xarg.org/2010/11/mysql-my-new-playground/";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:52:"http://www.xarg.org/2010/11/mysql-my-new-playground/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:609:"The business-logic should not be the task of a database server, because hardware of database servers is mostly more expensive than the sheet metal of some stupid application servers. The other problem are inconsistencies if you change the code-base on a large database cluster very often. You could reduce the risk of inconsistency with batches, but that does not solve the problem completely. However, I've developed a handful of stored functions to simplify queries, reduce index sizes and to shorten strings where only a small portion is needed, which in turn saves a lot of bandwidth.Read the rest »";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 00:57:40 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:3:"UDF";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"function";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:8:"wishlist";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:11:"performance";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:981:"The business-logic should not be the task of a database server, because hardware of database servers is mostly more expensive than the sheet metal of some stupid application servers. The other problem are inconsistencies if you change the code-base on a large database cluster very often. You could reduce the risk of inconsistency with batches, but that does not solve the problem completely. However, I've developed a handful of stored functions to simplify queries, reduce index sizes and to shorten strings where only a small portion is needed, which in turn saves a lot of bandwidth.
Read the rest »
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Robert Eisele";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:42;a:6:{s:4:"data";s:78:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:27:"Kettle vs Oracle REF CURSOR";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:28:"http://www.ibridge.be/?p=191";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:28:"http://www.ibridge.be/?p=191";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1447:"Dear Kettle fans,
PDI-200 has been out there for a while now. Jens created the feature request a little over 3 years ago. I guess the main thing blocking this issue was not as much a technical problem but more of a licensing and dependency one (Oracle JDBC dependency and distribution license).
However, now that we have the User Defined Java Class step we can work around those pesky problems. That is because the Java code in there only gets compiled and executed at runtime so it’s perfectly fine to create any sort of dependency in there you like.
The following transformation reads a set of rows from a stored procedure as described on this web page.
In short, our UDJC step executes the following code:
begin ? := sp_get_stocks(?); end;
The result is a set of rows and the parameter is in this case a single numeric value.
The step contains mostly Java code but thanks to configuration options you only need to do 2 things to make this work for your own REF CURSOR returning procedures…
First you need to specify the output fields of the rows…
And then you need to specify the parameters:
The source code for this sample transformation is over here and runs on Pentaho Data Integration version 4.x (or higher). All in all it only took a few hours to write these 150 lines of Java so perhaps it can serve as inspiration for other similar problems you might have with Oracle or other databases.
Until next time,
Matt";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 17 Nov 2010 00:01:40 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:16:"Data Integration";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"Kettle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:3:"PDI";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:7:"PDI-200";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:7:"pentaho";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:10:"REF Cursor";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2319:"Dear Kettle fans,
PDI-200 has been out there for a while now. Jens created the feature request a little over 3 years ago. I guess the main thing blocking this issue was not as much a technical problem but more of a licensing and dependency one (Oracle JDBC dependency and distribution license).
However, now that we have the User Defined Java Class step we can work around those pesky problems. That is because the Java code in there only gets compiled and executed at runtime so it’s perfectly fine to create any sort of dependency in there you like.
The following transformation reads a set of rows from a stored procedure as described on this web page.
In short, our UDJC step executes the following code:
begin ? := sp_get_stocks(?); end;
The result is a set of rows and the parameter is in this case a single numeric value.
The step contains mostly Java code but thanks to configuration options you only need to do 2 things to make this work for your own REF CURSOR returning procedures…
First you need to specify the output fields of the rows…
And then you need to specify the parameters:
The source code for this sample transformation is over here and runs on Pentaho Data Integration version 4.x (or higher). All in all it only took a few hours to write these 150 lines of Java so perhaps it can serve as inspiration for other similar problems you might have with Oracle or other databases.
Until next time,
Matt
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Matt Casters";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:43;a:6:{s:4:"data";s:38:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:55:"O'Grady's Fear of Forking, Let a thousand flowers bloom";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:39:"http://krow.livejournal.com/698497.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:39:"http://krow.livejournal.com/698497.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3616:"In the article "Fear of Forking" there was a quote pulled from me about my observations from a yearly call done by the folks at O'Reilly with many of the authors of different open source projects."On a related note there was a recent phone call that O’Reilly put together with a number of open source leads. It was amazing to hear how many folks on the call where terrified of how Github has lowered the bar for forking. Their fear being a loss of patches. It was crazy to listen too."Since I made that comment, there is one new observation I have made. GitHUB has begun to feel like the Sourceforge of the distributed revision control world. It feels like it is littered with half started, never completed, or just never merged trees. If you can easily takes changes from the main tree, the incentive to have your tree merged back into the canonical tree is low. I feel like you can look at it in either two ways. If you count up all of the hours and energy going into abandoned trees then you begin to worry about "all of that wasted work". It takes a lot of effort to keep projects going, and if all new energy is focused in this direction I don't know that we can keep a sustainable amount of focus to produce the sort of software that we do today. While consulting this last year I've run into a number of shops where a developer has made changes to an open source project, and placed these into production without any vetting (and in most of these cases they had a github/launchpad/etc sort of tree, or they pulled from some random person's tree). They didn't use a released piece of software, and often the code they had used was just thrown over the wall by some devs in some other company. It is the "we hired a smart guy who tinkered with our debian distribution/kernel" problem all over again.The other way to look at it, is that Github/Launchpad are today's Burgess Shale. We are in the equivalent of a cambrian explosion and the diversification we are seeing is similar to what we saw when Sourceforge first launched. If this is the case then we will see some stabilization in the next few years. In the database world, we are certainly in the middle of one of these periods. If I put all of this into perspective and apply it to the MySQL Ecosystem, I fully believe that the forking we saw was enabled by the move to bzr/launchpad. Without that move it would have been a lot harder to make that shift for most of the forks and distributions (and I believe it has also slowed down the evolution of most, since almost all of the forks/distributions are heavily tied to downstream changes that Oracle makes). Beyond Drizzle, none of the other forks have any significant contributions, and they are all stuck waiting for Oracle to fix bugs for them and/or hoping that the changes they make don't conflict with what Oracle is doing.In a related Ecosystem, I am eager to see what happens in the Postgres world now that they have moved to Git. As I have mentioned before, with Drizzle we could have started with Postgres as the foundation, and I believe there would have been a lot of benefit in doing so. I will be curious to see if anyone decides to see what they could do with the code if they take a radical departure from the current architecture. I am still happy with our choice of MySQL, but I believe there is an opportunity to do something pretty incredible with that codebase as well.In the cloud world we have OpenStack, Canonical, and Eucalyptus all circling around the same problem and having a history of shared tools and code. It is going to be interesting to see what happens there as well.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 19:19:05 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4024:"In the article "Fear of Forking" there was a quote pulled from me about my observations from a yearly call done by the folks at O'Reilly with many of the authors of different open source projects.
"On a related note there was a recent phone call that O’Reilly put together with a number of open source leads. It was amazing to hear how many folks on the call where terrified of how Github has lowered the bar for forking. Their fear being a loss of patches. It was crazy to listen too."
Since I made that comment, there is one new observation I have made. GitHUB has begun to feel like the Sourceforge of the distributed revision control world. It feels like it is littered with half started, never completed, or just never merged trees. If you can easily takes changes from the main tree, the incentive to have your tree merged back into the canonical tree is low.
I feel like you can look at it in either two ways.
If you count up all of the hours and energy going into abandoned trees then you begin to worry about "all of that wasted work". It takes a lot of effort to keep projects going, and if all new energy is focused in this direction I don't know that we can keep a sustainable amount of focus to produce the sort of software that we do today. While consulting this last year I've run into a number of shops where a developer has made changes to an open source project, and placed these into production without any vetting (and in most of these cases they had a github/launchpad/etc sort of tree, or they pulled from some random person's tree). They didn't use a released piece of software, and often the code they had used was just thrown over the wall by some devs in some other company. It is the "we hired a smart guy who tinkered with our debian distribution/kernel" problem all over again.
The other way to look at it, is that Github/Launchpad are today's Burgess Shale. We are in the equivalent of a cambrian explosion and the diversification we are seeing is similar to what we saw when Sourceforge first launched. If this is the case then we will see some stabilization in the next few years. In the database world, we are certainly in the middle of one of these periods.
If I put all of this into perspective and apply it to the MySQL Ecosystem, I fully believe that the forking we saw was enabled by the move to bzr/launchpad. Without that move it would have been a lot harder to make that shift for most of the forks and distributions (and I believe it has also slowed down the evolution of most, since almost all of the forks/distributions are heavily tied to downstream changes that Oracle makes). Beyond Drizzle, none of the other forks have any significant contributions, and they are all stuck waiting for Oracle to fix bugs for them and/or hoping that the changes they make don't conflict with what Oracle is doing.
In a related Ecosystem, I am eager to see what happens in the Postgres world now that they have moved to Git. As I have mentioned before, with Drizzle we could have started with Postgres as the foundation, and I believe there would have been a lot of benefit in doing so. I will be curious to see if anyone decides to see what they could do with the code if they take a radical departure from the current architecture. I am still happy with our choice of MySQL, but I believe there is an opportunity to do something pretty incredible with that codebase as well.
In the cloud world we have OpenStack, Canonical, and Eucalyptus all circling around the same problem and having a history of shared tools and code. It is going to be interesting to see what happens there as well.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:10:"Brian Aker";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:44;a:6:{s:4:"data";s:53:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:50:"Percona white paper: Forecasting MySQL Scalability";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=4031";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:97:"http://www.mysqlperformanceblog.com/2010/11/16/percona-white-paper-forecasting-mysql-scalability/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1412:"Ewen and I have just published Percona’s latest white paper, Forecasting MySQL Scalability with the Universal Scalability Law. This is essentially a streamlined walk-through of Dr. Neil J. Gunther’s book Guerrilla Capacity Planning, with examples to show how you can apply it to MySQL servers.
One thing alluded to in the paper is extracting the necessary metrics from network traffic. I had this idea after studying the data in Linux’s /proc/diskstats file. It turns out that two simple metrics can provide amazingly rich insight into system performance and scalability, in combination with Little’s Law and queueing theory. These are the busy time and the total time that requests were resident in the system. There are different terms for the latter, but in MySQL we’d call it query response time. After studying these for a few months, I’m so awed by how useful they are that I am going to make a blanket recommendation: if you create server software, you must expose these simple metrics. (I have filed a feature request for Percona Server to add these metrics for MySQL users.)
I’ll probably follow this up with another blog post or white paper at some point in the future, to show how to use the busy time and query response time to predict a system’s scalability.
Entry posted by Baron Schwartz |
3 comments
Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 18:07:26 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:8:"announce";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:11:"Scalability";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3838:"Ewen and I have just published Percona’s latest white paper, Forecasting MySQL Scalability with the Universal Scalability Law. This is essentially a streamlined walk-through of Dr. Neil J. Gunther’s book Guerrilla Capacity Planning, with examples to show how you can apply it to MySQL servers.
One thing alluded to in the paper is extracting the necessary metrics from network traffic. I had this idea after studying the data in Linux’s /proc/diskstats file. It turns out that two simple metrics can provide amazingly rich insight into system performance and scalability, in combination with Little’s Law and queueing theory. These are the busy time and the total time that requests were resident in the system. There are different terms for the latter, but in MySQL we’d call it query response time. After studying these for a few months, I’m so awed by how useful they are that I am going to make a blanket recommendation: if you create server software, you must expose these simple metrics. (I have filed a feature request for Percona Server to add these metrics for MySQL users.)
I’ll probably follow this up with another blog post or white paper at some point in the future, to show how to use the busy time and query response time to predict a system’s scalability.
Entry posted by Baron Schwartz |
3 comments
Add to: | | | |
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:45;a:6:{s:4:"data";s:68:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:57:"The perils of uniform hardware and RAID auto-learn cycles";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=4026";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:108:"http://www.mysqlperformanceblog.com/2010/11/16/on-the-perils-of-uniform-hardware-and-raid-auto-learn-cycles/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3515:"Last night a customer had an emergency in selected machines on a large cluster of quite uniform database servers. Some of the servers were slowing down in a very puzzling way over a short time span (a couple of hours). Queries were taking multiple seconds to execute instead of being practically instantaneous. But nothing seemed to have changed. No increase in traffic, no code changes, nothing. Some of the servers got sick and then well again; others that seemed to be identical were still sick; others hadn’t shown any symptoms yet. The servers were in master-master pairs, and the reads were being served entirely from the passive machine in the pair. The servers that were having trouble were the active ones, those accepting writes.
The customer had graphs of system metrics (++customer), and the symptoms we observed were that Open_tables went sharply up, the buffer pool got more dirty pages, InnoDB started creating more pages, and the percentage of dirty pages in the buffer pool went up. In addition, samples of SHOW INNODB STATUS showed a long pending flush list (hundreds), and usually one pending pwrite.
After talking through things for a bit, I mentally eliminated a number of possibilities. For example, there was no shared storage; servers were on different network segments and in different racks; and so on. I had two remaining possibilities.
The data had changed subtly, pushing it over some threshold. I’ve seen this before — for example, a query cache bottleneck, or an insert buffer bottleneck, or any of a variety of other things.
Rather than looking for differences between the servers, it might be fruitful to look at what was uniform about them.
The next step would be to measure precisely what was happening, but before even doing that, I wanted to chase down a small hunch that would only take a minute. I asked the customer whether the servers were all the same. Were they in the same shipment from the manufacturer? Did they have hard drives from a single manufacturing batch? It turned out that the servers were all part of a single shipment, and not only that, but they had been rebooted simultaneously pretty recently. I asked the customer to check the RAID controller quickly on one of the failing machines.
The result: the battery-backed cache was in write-through mode instead of write-back mode, and the battery was at 33% and charging. We quickly checked a few other sick machines. Same thing — write-through, battery charging. And the battery backup unit was set to run through auto-learn cycles periodically, which is generally the factory default (these are Dells, with PERC / LSI MegaRAID controllers.)
I opened my quick-reference text file and found the command we needed to disable auto-learn, (++text files and saving links to reference material), sent that over the Skype chat, and the customer tried that. We weren’t sure the auto-learn could be stopped in the middle, but it worked. They hung up and hurried to push this out with Puppet before it hit hundreds of other machines.
I have heard stories before of uniform hardware causing a cluster of problems, such as a bad batch of hard drives all failing close together, but this is the first time that I remember being involved in such a case myself. It was kind of fun that I didn’t actually have to get all measurement-y and scientific about the problem after all.
Entry posted by Baron Schwartz |
15 comments
Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 16:40:56 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:9:"AutoLearn";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:4:"Dell";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"MegaCLI";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:8:"MegaRAID";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:4:"PERC";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:6060:"Last night a customer had an emergency in selected machines on a large cluster of quite uniform database servers. Some of the servers were slowing down in a very puzzling way over a short time span (a couple of hours). Queries were taking multiple seconds to execute instead of being practically instantaneous. But nothing seemed to have changed. No increase in traffic, no code changes, nothing. Some of the servers got sick and then well again; others that seemed to be identical were still sick; others hadn’t shown any symptoms yet. The servers were in master-master pairs, and the reads were being served entirely from the passive machine in the pair. The servers that were having trouble were the active ones, those accepting writes.
The customer had graphs of system metrics (++customer), and the symptoms we observed were that Open_tables went sharply up, the buffer pool got more dirty pages, InnoDB started creating more pages, and the percentage of dirty pages in the buffer pool went up. In addition, samples of SHOW INNODB STATUS showed a long pending flush list (hundreds), and usually one pending pwrite.
After talking through things for a bit, I mentally eliminated a number of possibilities. For example, there was no shared storage; servers were on different network segments and in different racks; and so on. I had two remaining possibilities.
- The data had changed subtly, pushing it over some threshold. I’ve seen this before — for example, a query cache bottleneck, or an insert buffer bottleneck, or any of a variety of other things.
- Rather than looking for differences between the servers, it might be fruitful to look at what was uniform about them.
The next step would be to measure precisely what was happening, but before even doing that, I wanted to chase down a small hunch that would only take a minute. I asked the customer whether the servers were all the same. Were they in the same shipment from the manufacturer? Did they have hard drives from a single manufacturing batch? It turned out that the servers were all part of a single shipment, and not only that, but they had been rebooted simultaneously pretty recently. I asked the customer to check the RAID controller quickly on one of the failing machines.
The result: the battery-backed cache was in write-through mode instead of write-back mode, and the battery was at 33% and charging. We quickly checked a few other sick machines. Same thing — write-through, battery charging. And the battery backup unit was set to run through auto-learn cycles periodically, which is generally the factory default (these are Dells, with PERC / LSI MegaRAID controllers.)
I opened my quick-reference text file and found the command we needed to disable auto-learn, (++text files and saving links to reference material), sent that over the Skype chat, and the customer tried that. We weren’t sure the auto-learn could be stopped in the middle, but it worked. They hung up and hurried to push this out with Puppet before it hit hundreds of other machines.
I have heard stories before of uniform hardware causing a cluster of problems, such as a bad batch of hard drives all failing close together, but this is the first time that I remember being involved in such a case myself. It was kind of fun that I didn’t actually have to get all measurement-y and scientific about the problem after all.
Entry posted by Baron Schwartz |
15 comments
Add to: | | | |
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:46;a:6:{s:4:"data";s:78:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:71:"The 4 Things You Should Know When Designing Your Database for the Cloud";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:30:"http://blog.xeround.com/?p=174";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:107:"http://blog.xeround.com/2010/11/16/the-4-things-you-should-know-when-designing-your-database-for-the-cloud/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2097:"In his recent column, CNET’s Dave Rosenberg asks “Are databases in the cloud really all that different?”. The short answer: Yes, They ARE!
Addressing Rosenberg’s point, I described the 4 key issues every system architect should take into account when designing their database for the cloud — and why achieving them on a Cloud environment might be more complex, tedious to manage, and way more expensive than you might think.
Basically, you need:
Replicable, high-availability set-up - because the cloud is an unstable, unpredictable, environment.
Database that scales elastically - and scaling out by adding nodes is harder on a cloud infrastructure.
Distributed databases – which require building the logic to handle conflicts, network and latency problems.
True Multi-tenancy - And not a simple standard SQL database installed in multiple copies on the same virtual machine – which can cause more headaches and management overhead to keep running.
This is all hard work – we know..
In his review, Rosenberg also describes why relational databases are here to stay, and why NoSQL isn’t necessarily the answer for your cloud concerns.
At Xeround, we’ve always felt the answer lies in making SQL Cloud-able – for REAL! Our technology for managing data in the cloud provides both the transactional and query capabilities of relational databases, along with the simplicity and scalability as of NoSQL data stores.
Xeround SQL Database, offered for both public clouds (IaaS) and private clouds, was designed specifically for the cloud, to give you multi-tenancy, high-availability, auto-scaling and self-healing – all without being bogged down by the development, management and expenses overhead.
If you haven’t tried us out yet- sign up for the private beta and check it out for yourself!
You can read more on CNET’s Are databases in the cloud really all that different?
Filed under: Availability, Cloud Computing, Databases, MySQL Cloud Database, News & Press Mentions, NoSQL, Scalability, SQL ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 13:38:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:12:"Availability";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:15:"Cloud Computing";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:20:"MySQL Cloud Database";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:25:"News & Press Mentions";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:5:"NoSQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:11:"Scalability";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:3:"SQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5055:"In his recent column, CNET’s Dave Rosenberg asks “Are databases in the cloud really all that different?”. The short answer: Yes, They ARE!
Addressing Rosenberg’s point, I described the 4 key issues every system architect should take into account when designing their database for the cloud — and why achieving them on a Cloud environment might be more complex, tedious to manage, and way more expensive than you might think.
Basically, you need:
- Replicable, high-availability set-up - because the cloud is an unstable, unpredictable, environment.
- Database that scales elastically - and scaling out by adding nodes is harder on a cloud infrastructure.
- Distributed databases – which require building the logic to handle conflicts, network and latency problems.
- True Multi-tenancy - And not a simple standard SQL database installed in multiple copies on the same virtual machine – which can cause more headaches and management overhead to keep running.
This is all hard work – we know..
In his review, Rosenberg also describes why relational databases are here to stay, and why NoSQL isn’t necessarily the answer for your cloud concerns.
At Xeround, we’ve always felt the answer lies in making SQL Cloud-able – for REAL! Our technology for managing data in the cloud provides both the transactional and query capabilities of relational databases, along with the simplicity and scalability as of NoSQL data stores.
Xeround SQL Database, offered for both public clouds (IaaS) and private clouds, was designed specifically for the cloud, to give you multi-tenancy, high-availability, auto-scaling and self-healing – all without being bogged down by the development, management and expenses overhead.
If you haven’t tried us out yet- sign up for the private beta and check it out for yourself!
You can read more on CNET’s Are databases in the cloud really all that different?
Filed under: Availability, Cloud Computing, Databases, MySQL Cloud Database, News & Press Mentions, NoSQL, Scalability, SQL 
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Avigail Ofer";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:47;a:6:{s:4:"data";s:43:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:52:"MySQL Performance: 5.5 and InnoDB thread concurrency";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:68:"http://dimitrik.free.fr/blog/archives/11-01-2010_11-30-2010.html#116";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:68:"http://dimitrik.free.fr/blog/archives/11-01-2010_11-30-2010.html#116";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:21117:"
Until now, when you observed a performance degradation with a growing
number of concurrent user sessions in InnoDB the most simple way to
avoid such a degradation was to set innodb_thread_concurrency
to something different from zero (zero is a default now in 5.5) - and
since 5.5.4 the most optimal value was around of 32 (before
it was 16 )... But all latest performance improvements in 5.5 are so
good that it become less and less true now, and during my last tests
I've even observed a worse performance when a different from zero
"innodb_thread_concurrency" setting was used.. - seems to me now the
overhead of the thread concurrency management code in some cases become
more important than an overhead of still existing lock contentions ;-)
Let's see more in details now..
First of all - why "innodb_thread_concurrency" is even existing as
parameter?.. - MySQL is written as a multi-threaded application and each
connected user session is a thread within MySQL (and OS as well), which
is quite more optimal comparing to other vendors where a session is an
independent process (like in PostgreSQL or Oracle) just because a thread
is way more lightweight for OS comparing to a process, and context
switching between threads is usually going much faster and cost much
less comparing to processes based model.. - But! having a more optimal
model doesn't yet mean to have a scalable database engine! - and as I
say usually, everything depends on implementation ;-) And in
multi-threaded application all threads are running within the same
process and have direct access to everything inside of this process,
they may change any data at any time, so you have to apply and follow
some synchronization rules within your code to protect your data by
shared or exclusive locks to be sure there will be not possible to have
more than one changes of any critical data on the same time - and it has
a cost! :-) But again, all depends on how these locks are implemented
and used within a code.. And not so far, just 2 years ago(!)
MySQL/InnoDB did not scale more then up to 2-4 CPU cores! :-) - Today
the limit is already pushed up to 32 cores, and work is in progress ;-)
But what about InnoDB thread concurrency?..
InnoDB is also implemented as a multi-threaded application and also has
a lot of locks ;-) And once locking become too hot, the system start to
spend more CPU time to manage InnoDB thread locks rather to do any
useful work.. And these locks are not "user application locks" (for ex.
when many users are trying to modify the same raw) - no, these locks are
coming from InnoDB implementation design and independent to your "user
application" - and once you're blocked by InnoDB internal locks, you're
done.. ;-) (except if you feel yourself comfortable to modify the
MySQL/InnoDB code :-))
Ok, but even if this lock contention is inevitable, may it be at least
reduced or limited?..
Yes, it's possible - all you need is just to find after which amount of
threads running concurrently you're observing a performance degradation,
and then simply limit the number of possible active threads to this
amount! And there are at least two possible solutions:
Use "innodb_thread_concurrency=N" which will limit the number of
active threads within InnoDB exactly to N
Place MySQL process into a processor set (Solaris) or task set (Linux)
and limit it to N CPU cores
Both solutions are not free from limitations, but in the past the
first one was the most optimal.. Then things are changed with a time,
MySQL/InnoDB became more optimal and if you use Linux - it's also
improving all the time :-)
So if in the past, on most of my tests setting
"innodb_thread_concurrency=16" improved
performance stability since 32 concurrent user sessions and looked
like The MUST... Now, I have a 32 cores (bi-thread) Intel box running
Linux, and let's see if things are looking differently TODAY ;-)
I want to compare the same workload within the following
configurations :
#1) 32 cores, no task set, innodb_thread_concurrency=0
#2) MySQL is running within an 8 cores task set,
innodb_thread_concurrency=0
#3) MySQL is running within an 16 cores task set,
innodb_thread_concurrency=0
#4) 32 cores, no task set, innodb_thread_concurrency=8
#5) 32 cores, no task set, innodb_thread_concurrency=16
#6) 32 cores, no task set, innodb_thread_concurrency=32
NOTE : task set on Linux is not giving any exclusivity on the
task-set CPU cores like processor-set on Solaris, so my load generator
may use these cores too and it's not a problem here, because my goal is
to reduce the MySQL/InnoDB contention by reducing the number of active
tasks executed in parallel.. - Also, usually when we're presenting
results on 2/ 4/ 8/ 16/ cores we're also limiting the injector to
something equal.. - I'm not limiting injector here, I want to have it
running on its full power, and just limit the contention within MySQL,
so a test result on 8 cores task set doesn't mean a MySQL result on 8
cores! - just that MySQL contention is limited to 8 cores and other
cores may still do other useful stuff (networking, I/O, etc. ;-))
Also - my CPU cores are having 2 hardware threads, so it's hard to say
that one core is equal to one simple core, in case of MySQL bi-thread
core may work as well as two cores!.. - and helping to reduce the lock
contention as only one hardware thread from two may be executed by its
core on the same time..
Testing Scenario
I'll start with a Read-Only workload first (and as the Read seems to
drive a response time, Read+Write is hardly depending on the Read
performance).. As the lock contention may greatly be dependent on the
SQL query nature, I'll test at least 3 queries having a different
contention pattern:
Queries :
Query-1: main contention on "btr_search_latch" mutex
Query-2: main contention on the "buffer pool mutex" and "kernel_mutex"
Query-3: main contention on the "kernel_mutex"
Workload :
Users: growing from 1 to 256 (step by step: 1, 2, 4, 8, .. 256)
Think time between transactions: 0 ms
To save place here, I'm putting the results from all 6 tested
configurations on the same graph. Each configuration was tested one
after each other, so you may observe 6 workloads over a time scale per
graph. The first graph representing the number of active sessions
(users). The second one - reached TPS level. And the third one - Top-10
Mutex Waits reported by InnoDB.
NOTE : the code managing the InnoDB thread concurrency is also
using a mutex, but the waits on this mutex are not reported by InnoDB
(BUG ??), so I may only suppose that this mutex become the hottest when
the "innodb_thread_concurrency" set to a different from zero value (and
as you understand, it cannot be seen either by Performance Schema if
InnoDB is not reporting it..)
Let's get a look on the results now..
Query-1 @MySQL 5.5.6
Workload scalability on this SQL query is mainly limited by contention
on the "btr_search_latch" mutex.
Observations :
Configuration with a task set on 8 cores seems to be the most optimal
;-) (BTW, I've also tested 2 and 4 cores task sets, the results is way
worse, so I don't present these results here and you may just trust me
here :-))
Using non-zero innodb_thread_concurrency giving a lower results even
on 256 concurrent users comparing to default
innodb_thread_concurrency=0 setting..
Interesting that in last 3 configurations the mutex waits are reduced
dramatically! - however the waits on the concurrency mutex are not
reported by InnoDB and I suppose these waits are the main stopper
here, so we don't see an improvement similar to seen on 8 cores task
set..
Query-2 @MySQL 5.5.6
This query is limited by the "buffer pool mutex" and "the kernel_mutex"
contentions.
Observations :
Similar to Query-1, the highest and the most stable performance level
is obtained on the 8 cores task set configuration..
Again, limiting lock contention via "innodb_thread_concurrency" makes
things worse..
Query-3 @MySQL 5.5.6
This query is manly limited by contention on the "kernel_mutex".
Observations :
Performance on the 8 cores task set is the best once again..
However, on 32 cores having innodb_thread_concurrency=16 looks better
than zero! - seems contention on the "kernel_mutex" has a different
impact/pattern for this query..
Testing same 3 configurations on XtraDB 9.1
I was curious to replay the same workloads also on XtraDB 9.1 - but NOT
in goal to compare it with MySQL 5.5 (otherwise I have to use the latest
XtraDB version, and it's not the case) - my interest here is to see the
impact of the "innodb_thread_concurrency" setting but when the
"timer-based" concurrency model is enabled! - this model was introduced
within MySQL 5.4, ported by Percona team into XtraDB, but was finally
not included within MySQL 5.5. However on its time this model was near
free of locks, and as XtraDB 9.1 was comparable in performance with
MySQL 5.5.4, I was curious to know if we did not miss anything within
the latest releases of MySQL 5.5..
NOTE : during the following tests on XtraDB the task-set
configurations were tested in a different order - 8, 16, 32 cores vs
previously 32, 8, 16 cores.. - so just keep in mind when looking on the
graphs ;-) (sorry for that, but it was too late to fix it)..
Query-1 @XtraDB 9.1
Observations :
As you can see, the task set on 8 cores is still giving a better
results
However, using a Timer-Based concurrency model is not too bad, and
even showing the max TPS level!
Query-2 @XtraDB 9.1
Observations :
Wow! - Timer-Based model is a true winer here!
Interesting that Query-2 is running x2 times faster vs Query-1 on
MySQL 5.5, but not on XtraDB 9.1 don't know why.. - any ideas?..
As well curiously the highest mutex waits here are on the
"kernel_mutex", while on MySQL 5.5 it's the "buffer pool mutex".. -
seems to me it'll be always great to have 2 engines to replay the same
workload and better understand performance issues you have on your
application.. ;-)
Query-3 @XtraDB 9.1
Observations :
Curiously, on this query NOT only 8 cores task-set is keeping
workload stable and without degradation..
As well, the max TPS level is reached with a Timer-based model here..
Test with 1600 concurrent users
So far, until now from the previous tests globally the most better
results are obtained within an 8 cores task-set configuration.
But what will happen if I have more than 256 concurrent user sessions?..
- in the past this test was impossible to run successfully having
"innodb_thread_concurrency=0" whatever a number of CPU you have.. Well,
of course, a system having 1600 concurrent users is usually has not the
same workload profile as a non-stop 1-256 users tested before.. The main
difference here will be with a presence of a "think time", which means
that each user will do a pause between transactions (simulation of user
actions, ex. reading a web page, etc.).
Before I've executed 1600 users test with a think time=1sec, which means
that if everything is going well, each user should be able to process
one transaction per second, and with a growing number of concurrent
users we have to reach finally 1600 TPS. However, without limiting
InnoDB thread concurrency it was impossible to reach this level before,
and the following results were true just 2 years ago tested on different
MySQL versions:
Testing 1600 users with innodb_thread_concurrency=0 :
Testing 1600 users with innodb_thread_concurrency=8 :
All details about these 2 years old results you may find in this
report . As you can see, setting "inodb_thread_concurrency=8" solved
the situation and guaranteed a stability within an execution order! - so
each user was able to run on the one transaction/sec speed.
So, where we're now, 2 years later?.. ;-)
Few surprises..
Curiously reaching 1600 users sessions on the Linux box was not without
surprises..
First of all I've got a Connection Error messages (the issue details
an its fix I've described previous
post ), but more surprising was to see MySQL 5.5 simply crashing
in this situation while 5.1 and XtraDB printed an error message.. -
But once I've told about the problem to the MySQL team, the bug
was already fixed the next day by Davi ! - and it was my next
surprise as there was not yet even bug open nor test case, etc.. -
it's simply great to see such a reactivity! ;-)
Another problem seems to be related to the Linux kernel and I was
unable to fix it on time: since a high number of running processes the
network communications started to have timeouts, and some statistics
data were missed time to time (you may recognize this periods by
vertical red point lines on the following graphs)..
And the last surprise was that once I've start to test 1600 users
according to the same conditions as 2 years ago (think time=1sec) -
there is no difference at all whatever from 6 configurations I'm
testing!.. - so 1sec for MySQL 5.5 is too much now as a think time,
and I've reduced think time to 300ms to start to see any
significant difference between the tested configurations.. ;-))
So, what about results?..
Read-Only 1600 users @MySQL 5.5.6
Observations :
Hmm.. - once again the 8 cores task-set showing the best performance
here..
All 1600 users were active during every observed period
However since 1000 users (in best case) not every user was able to
execute its 3 transactions/sec, but TPS remains stable around 3000..
Read+Update 1600 users @MySQL 5.5.6
Observations :
On the last test I wanted to see the impact of the presence of the
Write transactions mixed to Reads.. - only Update statements are used
within a Write transaction here and the ratio between Read/writes is
50/50 here..
Surprisingly, random Updates are not slowing down MySQL performance on
this test - it's simply executing 2 times more transactions!.. -
contention on Reads seems to be the main blocking factor here
Also interesting that the level of the mutex waits did not change with
the presence of Updates - still the same contention on the
"btr_search_latch" and "kernel_mutex"..
Once again, the 8 cores task-set shows the most better result..
Instead of SUMMARY.. ;-)
Seems to me since MySQL 5.5.6 the "innofb_thread_concurrency" setting
should be used now very carefully! - as I was able to see several
cases when it makes things worse, then you'll be able too ;-) -
seriously, just keep in mind that it may reduce your
performance, so always check what's going on.. - what is fine at least
that this variable is dynamic and can be changed back at any time, so
it's easy to check if it helps you or not even on the live workload..
Using CPU Affinity - looks very attractive! and I
think there is definitely something to do! - this solution is
absolutely free of application locks, so may only work :-) then of
course it depends on how often do you spin on locks, etc. - but I
think just having a solution to assign Master and Purge threads to
dedicated CPU cores, and all "user" threads to other cores may help a
lot in heavy workloads! As well it can be dynamic, etc..
Timer-based concurrency model - is still looking interesting and gives
better results comparing to the original model. But as I wrote before,
it cannot guarantee CPU cycles for every thread just because threads
are waked up by timeout in a random order.. As well, it also as
original model doesn't have a solution for a case when an active
thread is involving an I/O operation and blocking on it for a while..
- during this time CPU cycles may be used by another thread from the
wait queue, but it's not so..
Thread Pool - honestly, this option combined with CPU Affinity
looks for me as the most attractive solution.. ;-)) But well,
when I'm speaking about a Thread Pool I don't mean the
"pool-of-thread" code proposed before within an MySQL 6.0 branch (once
Mark very well commented
about at the end of his article and I may only agree with him..) -
but I mean here a Thread Pool natively integrated into MySQL and aware
about everything what all the threads are doing: aware about
transaction locks, sleeps, I/O requests, etc. :-) - about all these
"internal events" CPU Affinity cannot be aware! - and according to
these events, activity and configuration settings Thread Pool should
limit optimally the number of the really active threads reduce missed
CPU cycles due wait events if there are other threads in the wait
queue.. - as well having less active threads requiring a CPU time will
simplify a lot the things for OS scheduler too.. - And of course it
should be a near free of locks!! :-)) - do I want to much?.. - don't
think so ;-) but if you have any other ideas - please, share!! :-))
Of course, having free of locks and highly scalable MySQL+InnoDB code
will resolve most of described problems! - but I think it's not yet for
tomorrow.. ;-) and anyway, even if you're using a scalable database
engine like Oracle you may still seek for a solution of connections pool
limiting the amount of active tasks on your server.. - so I'm voting for
an advanced Thread Pool for MySQL! :-)
Any comments are welcome! ;-)
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 13:27:57 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:24216:"
Until now, when you observed a performance degradation with a growing
number of concurrent user sessions in InnoDB the most simple way to
avoid such a degradation was to set innodb_thread_concurrency
to something different from zero (zero is a default now in 5.5) - and
since 5.5.4 the most optimal value was around of 32 (before
it was 16 )... But all latest performance improvements in 5.5 are so
good that it become less and less true now, and during my last tests
I've even observed a worse performance when a different from zero
"innodb_thread_concurrency" setting was used.. - seems to me now the
overhead of the thread concurrency management code in some cases become
more important than an overhead of still existing lock contentions ;-)
Let's see more in details now..
First of all - why "innodb_thread_concurrency" is even existing as
parameter?.. - MySQL is written as a multi-threaded application and each
connected user session is a thread within MySQL (and OS as well), which
is quite more optimal comparing to other vendors where a session is an
independent process (like in PostgreSQL or Oracle) just because a thread
is way more lightweight for OS comparing to a process, and context
switching between threads is usually going much faster and cost much
less comparing to processes based model.. - But! having a more optimal
model doesn't yet mean to have a scalable database engine! - and as I
say usually, everything depends on implementation ;-) And in
multi-threaded application all threads are running within the same
process and have direct access to everything inside of this process,
they may change any data at any time, so you have to apply and follow
some synchronization rules within your code to protect your data by
shared or exclusive locks to be sure there will be not possible to have
more than one changes of any critical data on the same time - and it has
a cost! :-) But again, all depends on how these locks are implemented
and used within a code.. And not so far, just 2 years ago(!)
MySQL/InnoDB did not scale more then up to 2-4 CPU cores! :-) - Today
the limit is already pushed up to 32 cores, and work is in progress ;-)
But what about InnoDB thread concurrency?..
InnoDB is also implemented as a multi-threaded application and also has
a lot of locks ;-) And once locking become too hot, the system start to
spend more CPU time to manage InnoDB thread locks rather to do any
useful work.. And these locks are not "user application locks" (for ex.
when many users are trying to modify the same raw) - no, these locks are
coming from InnoDB implementation design and independent to your "user
application" - and once you're blocked by InnoDB internal locks, you're
done.. ;-) (except if you feel yourself comfortable to modify the
MySQL/InnoDB code :-))
Ok, but even if this lock contention is inevitable, may it be at least
reduced or limited?..
Yes, it's possible - all you need is just to find after which amount of
threads running concurrently you're observing a performance degradation,
and then simply limit the number of possible active threads to this
amount! And there are at least two possible solutions:
-
Use "innodb_thread_concurrency=N" which will limit the number of
active threads within InnoDB exactly to N
-
Place MySQL process into a processor set (Solaris) or task set (Linux)
and limit it to N CPU cores
Both solutions are not free from limitations, but in the past the
first one was the most optimal.. Then things are changed with a time,
MySQL/InnoDB became more optimal and if you use Linux - it's also
improving all the time :-)
So if in the past, on most of my tests setting
"innodb_thread_concurrency=16" improved
performance stability since 32 concurrent user sessions and looked
like The MUST... Now, I have a 32 cores (bi-thread) Intel box running
Linux, and let's see if things are looking differently TODAY ;-)
I want to compare the same workload within the following
configurations :
-
#1) 32 cores, no task set, innodb_thread_concurrency=0
-
#2) MySQL is running within an 8 cores task set,
innodb_thread_concurrency=0
-
#3) MySQL is running within an 16 cores task set,
innodb_thread_concurrency=0
-
#4) 32 cores, no task set, innodb_thread_concurrency=8
-
#5) 32 cores, no task set, innodb_thread_concurrency=16
-
#6) 32 cores, no task set, innodb_thread_concurrency=32
NOTE : task set on Linux is not giving any exclusivity on the
task-set CPU cores like processor-set on Solaris, so my load generator
may use these cores too and it's not a problem here, because my goal is
to reduce the MySQL/InnoDB contention by reducing the number of active
tasks executed in parallel.. - Also, usually when we're presenting
results on 2/ 4/ 8/ 16/ cores we're also limiting the injector to
something equal.. - I'm not limiting injector here, I want to have it
running on its full power, and just limit the contention within MySQL,
so a test result on 8 cores task set doesn't mean a MySQL result on 8
cores! - just that MySQL contention is limited to 8 cores and other
cores may still do other useful stuff (networking, I/O, etc. ;-))
Also - my CPU cores are having 2 hardware threads, so it's hard to say
that one core is equal to one simple core, in case of MySQL bi-thread
core may work as well as two cores!.. - and helping to reduce the lock
contention as only one hardware thread from two may be executed by its
core on the same time..
Testing Scenario
I'll start with a Read-Only workload first (and as the Read seems to
drive a response time, Read+Write is hardly depending on the Read
performance).. As the lock contention may greatly be dependent on the
SQL query nature, I'll test at least 3 queries having a different
contention pattern:
Queries :
-
Query-1: main contention on "btr_search_latch" mutex
-
Query-2: main contention on the "buffer pool mutex" and "kernel_mutex"
-
Query-3: main contention on the "kernel_mutex"
Workload :
-
Users: growing from 1 to 256 (step by step: 1, 2, 4, 8, .. 256)
-
Think time between transactions: 0 ms
To save place here, I'm putting the results from all 6 tested
configurations on the same graph. Each configuration was tested one
after each other, so you may observe 6 workloads over a time scale per
graph. The first graph representing the number of active sessions
(users). The second one - reached TPS level. And the third one - Top-10
Mutex Waits reported by InnoDB.
NOTE : the code managing the InnoDB thread concurrency is also
using a mutex, but the waits on this mutex are not reported by InnoDB
(BUG ??), so I may only suppose that this mutex become the hottest when
the "innodb_thread_concurrency" set to a different from zero value (and
as you understand, it cannot be seen either by Performance Schema if
InnoDB is not reporting it..)
Let's get a look on the results now..
Query-1 @MySQL 5.5.6
Workload scalability on this SQL query is mainly limited by contention
on the "btr_search_latch" mutex.
Observations :
-
Configuration with a task set on 8 cores seems to be the most optimal
;-) (BTW, I've also tested 2 and 4 cores task sets, the results is way
worse, so I don't present these results here and you may just trust me
here :-))
-
Using non-zero innodb_thread_concurrency giving a lower results even
on 256 concurrent users comparing to default
innodb_thread_concurrency=0 setting..
-
Interesting that in last 3 configurations the mutex waits are reduced
dramatically! - however the waits on the concurrency mutex are not
reported by InnoDB and I suppose these waits are the main stopper
here, so we don't see an improvement similar to seen on 8 cores task
set..
Query-2 @MySQL 5.5.6
This query is limited by the "buffer pool mutex" and "the kernel_mutex"
contentions.
Observations :
-
Similar to Query-1, the highest and the most stable performance level
is obtained on the 8 cores task set configuration..
-
Again, limiting lock contention via "innodb_thread_concurrency" makes
things worse..
Query-3 @MySQL 5.5.6
This query is manly limited by contention on the "kernel_mutex".
Observations :
-
Performance on the 8 cores task set is the best once again..
-
However, on 32 cores having innodb_thread_concurrency=16 looks better
than zero! - seems contention on the "kernel_mutex" has a different
impact/pattern for this query..
Testing same 3 configurations on XtraDB 9.1
I was curious to replay the same workloads also on XtraDB 9.1 - but NOT
in goal to compare it with MySQL 5.5 (otherwise I have to use the latest
XtraDB version, and it's not the case) - my interest here is to see the
impact of the "innodb_thread_concurrency" setting but when the
"timer-based" concurrency model is enabled! - this model was introduced
within MySQL 5.4, ported by Percona team into XtraDB, but was finally
not included within MySQL 5.5. However on its time this model was near
free of locks, and as XtraDB 9.1 was comparable in performance with
MySQL 5.5.4, I was curious to know if we did not miss anything within
the latest releases of MySQL 5.5..
NOTE : during the following tests on XtraDB the task-set
configurations were tested in a different order - 8, 16, 32 cores vs
previously 32, 8, 16 cores.. - so just keep in mind when looking on the
graphs ;-) (sorry for that, but it was too late to fix it)..
Query-1 @XtraDB 9.1
Observations :
-
As you can see, the task set on 8 cores is still giving a better
results
-
However, using a Timer-Based concurrency model is not too bad, and
even showing the max TPS level!
Query-2 @XtraDB 9.1
Observations :
-
Wow! - Timer-Based model is a true winer here!
-
Interesting that Query-2 is running x2 times faster vs Query-1 on
MySQL 5.5, but not on XtraDB 9.1 don't know why.. - any ideas?..
-
As well curiously the highest mutex waits here are on the
"kernel_mutex", while on MySQL 5.5 it's the "buffer pool mutex".. -
seems to me it'll be always great to have 2 engines to replay the same
workload and better understand performance issues you have on your
application.. ;-)
Query-3 @XtraDB 9.1
Observations :
-
Curiously, on this query NOT only 8 cores task-set is keeping
workload stable and without degradation..
-
As well, the max TPS level is reached with a Timer-based model here..
Test with 1600 concurrent users
So far, until now from the previous tests globally the most better
results are obtained within an 8 cores task-set configuration.
But what will happen if I have more than 256 concurrent user sessions?..
- in the past this test was impossible to run successfully having
"innodb_thread_concurrency=0" whatever a number of CPU you have.. Well,
of course, a system having 1600 concurrent users is usually has not the
same workload profile as a non-stop 1-256 users tested before.. The main
difference here will be with a presence of a "think time", which means
that each user will do a pause between transactions (simulation of user
actions, ex. reading a web page, etc.).
Before I've executed 1600 users test with a think time=1sec, which means
that if everything is going well, each user should be able to process
one transaction per second, and with a growing number of concurrent
users we have to reach finally 1600 TPS. However, without limiting
InnoDB thread concurrency it was impossible to reach this level before,
and the following results were true just 2 years ago tested on different
MySQL versions:
Testing 1600 users with innodb_thread_concurrency=0 :
Testing 1600 users with innodb_thread_concurrency=8 :
All details about these 2 years old results you may find in this
report . As you can see, setting "inodb_thread_concurrency=8" solved
the situation and guaranteed a stability within an execution order! - so
each user was able to run on the one transaction/sec speed.
So, where we're now, 2 years later?.. ;-)
Few surprises..
Curiously reaching 1600 users sessions on the Linux box was not without
surprises..
-
First of all I've got a Connection Error messages (the issue details
an its fix I've described previous
post ), but more surprising was to see MySQL 5.5 simply crashing
in this situation while 5.1 and XtraDB printed an error message.. -
But once I've told about the problem to the MySQL team, the bug
was already fixed the next day by Davi ! - and it was my next
surprise as there was not yet even bug open nor test case, etc.. -
it's simply great to see such a reactivity! ;-)
-
Another problem seems to be related to the Linux kernel and I was
unable to fix it on time: since a high number of running processes the
network communications started to have timeouts, and some statistics
data were missed time to time (you may recognize this periods by
vertical red point lines on the following graphs)..
-
And the last surprise was that once I've start to test 1600 users
according to the same conditions as 2 years ago (think time=1sec) -
there is no difference at all whatever from 6 configurations I'm
testing!.. - so 1sec for MySQL 5.5 is too much now as a think time,
and I've reduced think time to 300ms to start to see any
significant difference between the tested configurations.. ;-))
So, what about results?..
Read-Only 1600 users @MySQL 5.5.6
Observations :
-
Hmm.. - once again the 8 cores task-set showing the best performance
here..
-
All 1600 users were active during every observed period
-
However since 1000 users (in best case) not every user was able to
execute its 3 transactions/sec, but TPS remains stable around 3000..
Read+Update 1600 users @MySQL 5.5.6
Observations :
-
On the last test I wanted to see the impact of the presence of the
Write transactions mixed to Reads.. - only Update statements are used
within a Write transaction here and the ratio between Read/writes is
50/50 here..
-
Surprisingly, random Updates are not slowing down MySQL performance on
this test - it's simply executing 2 times more transactions!.. -
contention on Reads seems to be the main blocking factor here
-
Also interesting that the level of the mutex waits did not change with
the presence of Updates - still the same contention on the
"btr_search_latch" and "kernel_mutex"..
-
Once again, the 8 cores task-set shows the most better result..
Instead of SUMMARY.. ;-)
-
Seems to me since MySQL 5.5.6 the "innofb_thread_concurrency" setting
should be used now very carefully! - as I was able to see several
cases when it makes things worse, then you'll be able too ;-) -
seriously, just keep in mind that it may reduce your
performance, so always check what's going on.. - what is fine at least
that this variable is dynamic and can be changed back at any time, so
it's easy to check if it helps you or not even on the live workload..
-
Using CPU Affinity - looks very attractive! and I
think there is definitely something to do! - this solution is
absolutely free of application locks, so may only work :-) then of
course it depends on how often do you spin on locks, etc. - but I
think just having a solution to assign Master and Purge threads to
dedicated CPU cores, and all "user" threads to other cores may help a
lot in heavy workloads! As well it can be dynamic, etc..
-
Timer-based concurrency model - is still looking interesting and gives
better results comparing to the original model. But as I wrote before,
it cannot guarantee CPU cycles for every thread just because threads
are waked up by timeout in a random order.. As well, it also as
original model doesn't have a solution for a case when an active
thread is involving an I/O operation and blocking on it for a while..
- during this time CPU cycles may be used by another thread from the
wait queue, but it's not so..
-
Thread Pool - honestly, this option combined with CPU Affinity
looks for me as the most attractive solution.. ;-)) But well,
when I'm speaking about a Thread Pool I don't mean the
"pool-of-thread" code proposed before within an MySQL 6.0 branch (once
Mark very well commented
about at the end of his article and I may only agree with him..) -
but I mean here a Thread Pool natively integrated into MySQL and aware
about everything what all the threads are doing: aware about
transaction locks, sleeps, I/O requests, etc. :-) - about all these
"internal events" CPU Affinity cannot be aware! - and according to
these events, activity and configuration settings Thread Pool should
limit optimally the number of the really active threads reduce missed
CPU cycles due wait events if there are other threads in the wait
queue.. - as well having less active threads requiring a CPU time will
simplify a lot the things for OS scheduler too.. - And of course it
should be a near free of locks!! :-)) - do I want to much?.. - don't
think so ;-) but if you have any other ideas - please, share!! :-))
Of course, having free of locks and highly scalable MySQL+InnoDB code
will resolve most of described problems! - but I think it's not yet for
tomorrow.. ;-) and anyway, even if you're using a scalable database
engine like Oracle you may still seek for a solution of connections pool
limiting the amount of active tasks on your server.. - so I'm voting for
an advanced Thread Pool for MySQL! :-)
Any comments are welcome! ;-)
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:17:"Dimitri Kravtchuk";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:48;a:6:{s:4:"data";s:78:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:31:"How to resize Innodb log files?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://www.ovaistariq.net/?p=474";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:61:"http://www.ovaistariq.net/474/how-to-resize-innodb-log-files/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:247:"Just in case you don't know resizing Innodb log file is not just simple as changing the value of innodb_log_file_size in the MySQL configuration file. But its even more simpler. Follow the steps in this post and you will be done with the resizing.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 13:03:30 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:6:"Innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"how to";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:16:"innodb resources";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:20:"innodb_log_file_size";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:8:"log file";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:13:"log file size";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:15:"resize log file";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:473:"Just in case you don't know resizing Innodb log file is not just simple as changing the value of innodb_log_file_size in the MySQL configuration file. But its even more simpler. Follow the steps in this post and you will be done with the resizing.
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Ovais Tariq";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:49;a:6:{s:4:"data";s:88:"
";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:40:"DBJ – MySQL Hotbackups with Xtrabackup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:34:"http://oracleopensource.com/?p=163";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:76:"http://oracleopensource.com/2010/11/16/dbj-mysql-hotbackups-with-xtrabackup/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:486:"The open-source xtrabackup tool from Percona brings much needed hot backup functionality to MySQL deployments. In this database journal article we discuss logical, cold, and hot backups, then explain how to use xtrabackup on your MyISAM, InnoDB, and XtraDB tables to create at-the-ready backups. We then take you through the step-by-step process to restore them, and even the process of point-in-time recovery too.
Database Journal – Hotbackups with Percona’s Xtrabackup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 16 Nov 2010 13:00:28 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:10:{i:0;a:5:{s:4:"data";s:6:"backup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:9:"always on";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:11:"cold backup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:10:"hot backup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:14:"logical backup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:11:"open-source";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:7:"percona";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:8;a:5:{s:4:"data";s:10:"xtrabackup";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:9;a:5:{s:4:"data";s:6:"xtradb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:805:"The open-source xtrabackup tool from Percona brings much needed hot backup functionality to MySQL deployments. In this database journal article we discuss logical, cold, and hot backups, then explain how to use xtrabackup on your MyISAM, InnoDB, and XtraDB tables to create at-the-ready backups. We then take you through the step-by-step process to restore them, and even the process of point-in-time recovery too.
Database Journal – Hotbackups with Percona’s Xtrabackup
PlanetMySQL Voting:
Vote UP /
Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:9:"Sean Hull";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}}}}}}}}}}}}s:4:"type";i:128;s:7:"headers";a:7:{s:4:"date";s:29:"Tue, 23 Nov 2010 00:39:33 GMT";s:6:"server";s:22:"Apache/2.2.15 (Fedora)";s:13:"last-modified";s:29:"Tue, 23 Nov 2010 00:31:27 GMT";s:13:"accept-ranges";s:5:"bytes";s:14:"content-length";s:6:"364316";s:10:"connection";s:5:"close";s:12:"content-type";s:8:"text/xml";}s:5:"build";s:14:"20090627192103";}
When it comes to cloud computing, people want to command nearly endless resources while still being able to deploy, run and consume a distributed solution anywhere and anytime. The reality, however, is not that simple.