text:
Virginia Memory is part of the online presence of the Library of Virginia, the state archives and reference library. The Library maintains vast and varied collections of print materials, manuscripts, archival records, newspapers, photographs and ephemera, maps and atlases, rare books, and fine art that tell the history of the commonwealth and its people.
The Library of Virginia and Virginia Memory provide several RSS Feeds for their patrons including new collections, events and happenings at the Library, and site updates.
Pretty self-explanatory. Use Google to search the Virginia Memory web site for features, collections, or whatever may interest you.
Having difficulty navigating the web site, or just curious? Our site index provides an alphabetical list of all pages of the web site.
Questions? Comments? Suggestions? Drop us a line! The Library of Virginia's knowledgeable staff is here to help.


text:
Style # 215711
AVAILABILITY: Unavailable for immediate shipment.
Dotted cutout panels at shoulders and a sexy fitted silhouette make this bebe bodycon a hot way to inject spring color into your closet. Try it with a pair of cork wedges and an oversized tote for an instant day look.
- 92% nylon, 8% spandex
- Center back to hem: 28" (71 cm)
- Machine wash according to care label
- Made in USA with imported materials
- Model is 5'9" and is wearing a US size S
Rated 0 out of 5
0-0 of 0

subset: commoncrawl


text:
Divini Rae >> See More Playmates
Galleries in Full HD!
Galleries in Full HD!
You'll go wild for Divini Rae, our Miss November 2003. Tall and svelte at 5'8", with blonde hair and blue eyes, Divini is often mistaken for a native Californian, but she comes to us from the untamed Alaskan wilderness. "I grew up in a village called Tanana," she s...
Get this complete pictorial in stunning resolution & HD video! View Full Gallery!


text:
Type: Sub-task
Status: Closed
Priority: Blocker
Resolution: Fixed
Affects Version/s: 2.0.3, 2.1, 2.4
Fix Version/s: 2.4
Component/s: Administration, Libraries, Performance
Testing Instructions:
Run daily statistics with major log entries for a day.
With more than 150 000 entries, daily statistics generation can take 2 hours to resolve.
Also run phpunit tests across 5 DB's
Affected Branches:MOODLE_20_STABLE, MOODLE_21_STABLE, MOODLE_24_STABLE
Fixed Branches:MOODLE_24_STABLE
Pull from Repository:
Pull Master Branch:
Pull Master Diff URL:
Daily statistics generation is really slow because SQL queries are not optimized properly. The propblem occurs principaly when you have a lots of entries in the log table (150 000 and more per day). We also observed that the server is responding slowly for users in Moodle while the daily statistics generation is running (Every inserts in log table take about 40-50 seconds).
December 5th cron log from our server
Running daily statistics gathering, starting at 1322974800:
................ finished until 1323061200: lundi 5 décembre 2011, 00:00 (in 6872 s)
...completed 1 days of statistics.
Gliffy Diagrams
Issue Links
- caused a regression
MDL-46773 Midnight failure in test_statslib_cron_daily
- Open
- has a non-specific relationship to
MDL-36410 Extend our DB API to support temporary tables usage to allow for creation of more efficiently performing but not often components
- Closed
- has been marked as being related by
MDL-18484 Statistics are impossibly slow
- Closed
MDL-36799 Cannot generate weekly and monthly statistics reports
- Closed
MDL-36416 test_statslib_get_action_names() has the hard coded actions used in logging, perhaps these could be constants in stats lib or collated somewhere
- Open
MDL-36411 Allow asserts for phpunit tests during setUp() stage for setup failures.
- Closed
- All
- Comments
- Work Log
- History
- Activity
- Links Hierarchy
Here's a patch for the bug. The point is to create a temporary table containing the logs per day with the right indexes.
Queries are taking 40-45 seconds instead of 1h30-2h to run.
Thanks for sharing that solution. This could have a significant impact and it looks like it could resolve a long chain of issues.
There are very many problems related to logging and stats, I personally think that we should prioritize this for the next dev cycle.
The proposed patch is not unfortunately compliant with our coding style, also it is not possible to modify table structure in stable branches, sorry.
Disregard my last comment posted here I just though of what I may not have yet tried.
Petr, are you sure about it be contrary to the coding style? I was informed that creating a scratch table was acceptable behavior and has been done in other parts of Moodle where it was useful. On that basis, I don't think the use of a scratch table should disqualify this patch. My testing is showing an approximately 20 times performance improvement from the patch. Specifically, I tested with data from a production site and it took 900 seconds to generate stats before patch and 45 seconds after the patch. I tested the results for discrepancies and the actual results were identical before and after patch.
Yes, changes in database structure are not allowed in stable branches.
The stats are slow because we are doing crazy things there - the biggest mistake is trying to find out type of operation for each row (read or write) from the action name. It is inaccurate and very slow - the obvious solution is to add new column into the log table and specify the type when adding log entries.
I believe the current logging/stats is not worth improving, somebody should imo redesign it completely.
Thanks for your contribution anyway.
This could be changed to use "create_temp_table" and "drop_temp_table" if that makes the code compliant. Those functions are used in standard core code. Would that work?
Hi Mike,
I thought about this but unfortunately it's not an option with the current queries because of a MySQL limitation
See http://dev.mysql.com/doc/refman/5.1/en/temporary-table-problems.html :
"You cannot refer to a TEMPORARY table more than once in the same query."
Maybe the queries could be modified to only have one reference each but it would need time that I don't have... Feel free to propose a patch if you want base on what I started
Shifting this over to the DEV backlog as it will require DB changes.
I think if this change can make an improvement, even only to Master, then we should pursue it for now. I don't think we're going to get a chance to re-write logging in the near future.
Hi Michael, I disagree with proposed workaround because it would be imo a lot better to change log table structure and completely eliminate unnecessary operations in stats processing. Reassigning back, sorry.
So, how soon can the log table structure be changed, and the stats processing be rewritten?
In the meantime, large scale Moodle users will lose site functionality whenever they try to use stats functions. The way it is now, the sites hang completely. This is not acceptable. Should we not promote workarounds to fix problems when time will not allow complete rewrites?
I do.
If someone can come up with a patch for master that
- adds the new field with the read/write type
- change logging code to make sure that the field is filled during upgrade, and always kept full after that whenever entries are added (possibly slow) or at least on a cron job.
then this might solve the problem for now and allow an improvement for 2.3, pending a bigger rewrite at some point in the future.
I don't think the new read/write field will improve performance as much as predicted. If I completely remove the subqueries that try "to find out type of operation for each row (read or write) from the action name" then I see an approximately 40% drop in processing time. That's a far cry from the 95% drop achieved using the temporary table. Additionally, all of that drop appears to be related to removing the subqueries rather than the "IN" clause, I have spent some time working out alternative queries that use the IN clauses and produces the same results without the subqueries and I get virtually identical performance improvements with or without the IN clauses.
So, Tyler, you are suggesting that the correct solution is still to use temporary tables? Is that correct?
I've attached a new patch that should resolve all currently outstanding issues. This patch uses 2 temporary tables to reach the same 95% reduction in execution time that the original solution had. It also incorporates the improvements to the stats queries themselves (including one that reduces the number of needed temporary tables from 3 to 2) that I was working on.
Martin / Michael / Petr - This new patch looks like a significant improvement. It uses temporary tables solely through the temp tables API. Can this be integrated? We'll happily put the patch into a git branch if that is the preferred way.
Hi. Mike and all.
We haven't forgotten about this.
I'm chasing this up this week.
Whoa, this is amazing, I've just implemented the same thing because we have production systems unable to finish the stats in days.
It really does work, because instead of scanning either course/user or time index (can't reasonably do both), pre-selecting the right times with course/user indexes allows the DB to scan only the needed bits.
thoughts on patch above...
- Don't need all columns. course, userid, action should be enough.
- Don't need all indexes, one for (course,userid) and one for (userid) worked nicely for me
- If temptables is a problem for mysql, maybe only 1 perm table first truncated is fine?
Ok, here is my contribution rebased to master featuring:
- Rewrite all queries for speed/scaling and to make only one reference to log_temp, generally simplified etc
- Pre-aggregate the log_temp table data by action and ignore log.course = 0 (seems bogus)
- Don't use stats_user_daily in the calculations, as PG is making horrific plans against it
I've ignored (and removed) the PG 8.2 warnings, as 8.3 is the oldest supported in stable by now AFAIK. These should in theory work against all 4 DBs because other core code uses the same constructs, but obviously this will need testing.
With the data I'm using - about 120-150k log entries per day - the daily stats are calculated in about 15 seconds consistently which is much nicer than the 'days, until killed' issue we're having at the moment.
Just updated with a commit to fix a case where the temptable isn't disposed.
Otherwise, this is working nicely on production so far.
Hi Tony, I found a few issues with your changes when I tested it on my test data:
- Your changes produced different numbers of results in stats_daily and stats_user_daily when compared to the original stats code.
- Some of the rows in stats_daily and stats_user_daily had slightly different totals when compared to the original stats code.
- Your patch is a little slower than the last patch I attached here (79 seconds versus 43 seconds).
- You have introduced a potential crash bug and the potential to generate incorrect stats. At around line 188 in your patch you conditionally generate the temporary table if there are logs present. However, you reference the temporary logs table at least once (at 249) without also checking that logs were present thus if the first day of results is empty, stats generation will crash because it attempted to access a table which doesn't yet exist. Similarly, if any day has no logs that query will generate results based on the previous day's logs.
Most of the additional time appears to be consumed by query 11 which calculates the number of guest accesses.
I've incorporated some of Tony's changes into an updated patch on github for the issue that is a little bit faster than the one I previously added. Strangely, when I changed the order of inserting data and adding indexes, the temporary table creation time tripled. I would have thought it would be faster to insert the data and add the indexes later but testing indicated otherwise. Similarly, I didn't keep the changes to the database queries because it looked to be nearly universally slower with the new temporary table format. I did however eliminate the unused columns and indexes from the temporary table and switch to use microtime instead of time for query tracking.
Which kind of rows are different? I tested with some weeks of production data so this is surprising.
Tyler - any further feedback on this? - Tony have you had a chance to compare the stats that Tylers patch generates against the stats your patch generates?
also Tyler - which db types have you tested your patch on? it appears Tony has been running on PG
I don't have the test setup anymore ... but I did compare my results to the existing core code, so if there is a difference to Tyler's patch it is either some different data my test didn't hit or that patch is bad.
My patch series may be slower in some conditions but the idea is that it should scale better and avoids bad queries against the stats daily tables, which is just a smaller version of the mdl_log problem (i.e it will sometimes pull things by the course, user or role index, which will get slower and slower as the amount of entries increases over time).
Switching /everything/ to use log_temp ensures a basically constant run time for years at a given traffic level. IMO, it is fair enough to get a bit slower with more traffic, but not OK to be dramatically slower over time.
There is one difference in (I think) stats_user_daily for course=0, userid=0 - the row appears to have no purpose so I haven't included it in the new version of the queries. It looks like a hack to make some of the later stats_daily things work but is now obsolete as everything runs though log temp. If any other rows are different I'd really like to know what they are and the data that causes it so it can be fixed...
I've fixed the only missing $logspresent check to prevent the 'crash' issue and rebased to master again.
please, try to consider also a 1.9 backporting to close
MDL-18484 too, when a fix will be available for 2.x.
Even though 1.9 will be EOLed everything about slow and blocking daily stats has been brought to the attention by 1.9 users
I'm testing with MySQL. Here are the differences I'm seeing for a month of results:
Pre-patch: 161,016 records
Post-patch: 160,964 records
It looks like I was mistaken about row differences in stats_daily, the tow totals appear to be same when I re-ran Tony's patch (excepting of course the duplicate rows (courseid=siteid,roleid=0) that both version of the stats code create).
The missing rows are curious, in this case, there appears to be about 50 missing rows all for a specific role (72 - Guest no profile) with courseid 1 or 0. The remaining two missing rows are duplicates that weren't created with Tony's patch.
Pre-patch: 34,312 records
Post-patch: 34,261 records
The missing rows for stats_user_daily are activity logs with courseid=0, roleid=0.
stats_user_daily also has one record which is different for each day (site activity for not logged in users), however, that appears to be a duplicate row that has been created with 0s in it. Thus there are 31 additional rows in Tony's results and 82 missing rows.
I see the point about avoiding using the stats_daily tables. However, it seems to me there will still see some scaling with the new queries according to number of courses, users, and site activity. It might be a good idea to investigate generating the daily stats directly into temporary tables and copying the results into the permanent tables when the day's stats have been completed. That should also reduce the risk of incomplete or incorrect stats being generated if the cron is interrupted.
@Matteo - There is already a similar patch attached to
MDL-18484 that would reduce log creation time by a factor of 60.
I've incorporated my idea about using temporary tables to avoid the potential impact of history growth on the daily tables into my last posted patch. The diff between the updated branch and master is available on git hub (https://github.com/tbannister/moodle/compare/master...MDL-30643_master-log-speed). This new patch is a little slower than the last one, the addition of the extra temporary tables seems to counteract the gains made by removing unnecessary tables and indexes. However, in theory, it should perform better on sites with with many entries in the daily stats tables.
Still not a fan; those updates against stats daily are the most expensive part because of the time spent in loops against it so this patch wouldn't help our case much.
Updating several stats for each operation on the site is no way ever going to work in high concurrency, unless I misunderstand?
Tony / Tyler, can you each summarize the cases you are concerned about, so we can see if there is a common solution?
Tony, I don't think I understand why it wouldn't help your case much. Could you explain a bit more about what your case is?
Also, I don't understand what you mean with your high concurrency comment. Since my last patch substitutes a temporary table (mdl_tmp_stats_daily and mdl_tmp_stats_user_daily) for each stats table, it should have the fewest possible concurrency issues. The real tables will only be accessed once for reading, and once for writing. The update queries will operate in isolation on those temporary tables because each process can only access it's own temporary tables.
my concern would be the testing across multiple db types (I haven't looked at the patch in detail) - but in the sql performance work I've done in the past I've seen massive differences in the way SQL performs across the different DB types - what might work well for MySql might be really slow on other db types. It would be nice to see some stats across the different db types if poss (I haven't got the time to do the testing myself)
The /worst/ problem for us has been the updates against mdl_stats*_daily for the second column and they get worse over time; using log_temp didn't improve these because of the time sunk into query nestloops on the stats tables. This could run through the night and most of a day, with the worst cases running for 3 days straight.
It looks like the patches bring the times to a sane level for both PG and MySQL at least; somebody else will have to try MS & oracle.
Tony, if I understand correctly, then if the patch (https://github.com/tbannister/moodle/compare/master...MDL-30643_master-log-speed) changes the updates so that they no longer run against the mdl_stats*_daily tables (which it does) then the patch should be acceptable?
I don't currently have access to either MSSQL or Oracle either.
I think adding another temporary table is not a clean solution; its a workaround for the terrible queries. IMO, proper solution is to rewrite for the more modern DB requirements in 2.x - but maybe upstream is not interested in such a large diff and your patches will be better for them.
I've just run this over 2 weeks of production data to compare /again/ and there is no difference besides the courseid=0 bogus entries so I really think the integrity concerns require more explanation / evidence.
Rebased my branch to master again.
Tony, I don't think it's a good idea to conflate different changes together. An efficiency fix should produce identical results before and after the fix. If you would like to change what results are produced, a separate issue should be opened for that.
The second log temporary table is simply to work around a MySQL limitation that prevents the table from being referenced multiple times in the same query. With the extra temporary table I see about a 40% reduction in statistics generation time versus the queries that cut the extra table out. Personally, I'm willing to go with "a workaround" that provides a substantial performance improvement.
Hi Tony / Tyler...
Trying to get this to some closure.
Tony - Rewriting the stats was discussed at the top of this issue. Everyone agreed that rewriting is the preferred solution, but that it would not happen any time soon. So, we need a compromise fix to prevent the large users from failing. Or, are you saying you already have a rewrite, when you say "but maybe upstream is not interested in such a large diff"?
From what I read here, the latest patch provided from Tyler has worked on all the systems tested (MySQL and Postgres), and has proven to be quite faster. There is concern about it using two temporary tables, but again, this improves the performance and should be a good solution until the rewrite can be done.
There is still an unresolved discrepancy in the alternate solution provided by Tony, where data results were not the same from the current code.
Please let me know if there are any incorrect statements above.
If there are not, I would recommend that we implement the latest patch from Tyler to get this issue resolved.
There is no discrepancy that can be demonstrated so I'd appreciate it if people stopped repeatedly saying that without showing the goods??
It doesn't impact us much by now but given it /has/ been rewritten I see no reason (yet) that effort can't be integrated. Otherwise... well, we'll keep this piece forked as long as needed.
also - there seems to be reference to Tyler and his patch about "duplicate rows" ?
if the old code or Tylers patch creates incorrect stats that should be fixed at the same time as a patch - it's not important for a new patch to generate exactly the same stats if the old stats are wrong.
Tony, if I offended you, I apologize. It was not intentional. What I am trying to do is narrow down which patch is appropriate to consider recommending for addition into core. Martin D. indicated he would get an appropriate patch in.
As I understand it, there are two patches here for consideration. One supplied by Tony and one supplied by Tyler. Tyler's apparently adds another 40% improvement in time over Tony's.
With respect to the discrepancy, Tyler indicated in one of his comments that he saw data discrepancies with Tony's patch. Tyler - can you confirm this and provide proof? If the discrepancy is really a fix to incorrect data in the original code, as Dan suggests, we can dismiss that issue.
Tony, you said:
"It doesn't impact us much by now but given it /has/ been rewritten I see no reason (yet) that effort can't be integrated. Otherwise... well, we'll keep this piece forked as long as needed."
Can you elaborate on this a bit? What is that has been rewritten? Are you referring to the patches attached, or something else? And what do you mean by "we'll keep this piece forked"? Are you referring to just your organization?
So, given that Tyler's patch offers even more performance improvements, why would we not recommend that over Tony's? Just looking for the reasons.
Well it is late but.. I think the existing queries can have many problem cases where planners are going to fall down, especially related to the excessive use of subqueries, updates against stats tables etc as workarounds to really old and no longer existing DB issues - this is what we hit on different instances, they all have slightly different issues. My patch /series/ builds on Tyler's temptable-only patch, rewriting these poor queries and cleaning up the code a bit too.
In his particular data set and DB, some of these needlessly complex queries are slightly faster. Ok, but... that doesn't make them correct or desirable. Since we're talking about a 1000x speedup anyway, I hardly think 5 seconds vs 3 seconds seriously matters over simpler codes with less edge cases / problems with other data. Upstream should have a reasonable set of queries that are all-round good, rather than best for only DB X, data Y.
Hope this explains more.
When I ran Tony's patch over production data from a client, I saw 50 missing rows for a special guest role created by the client in mdl_stats_daily and 2 missing duplicate rows. Additionally, Tony's patch created 31 additional rows and did not create 82 other rows in mdl_stats_user_daily that were created by the original stats code. That's the discrepancy that I see between the output from Tony's patch and the original stats code.
I don't know if the results produced by the original stats code are correct or incorrect. I think that should be a separate issue because there could be code in core, blocks, modules or other add ons that depends on the current behavior. Additionally, it could be a very long and involved discussion on what those results should be and that could delay implementation of the performance patch. I would move discussion of what the stats should produce to a different issue to isolate the performance issue from the correctness issue. Especially since the correctness issue could easily morph into a re-design of the logging and stats generation systems that could take many more months to implement.
There are also practical aspects to consider as well, because whoever does the testing could legitimately reject a performance patch that doesn't produce the same results as the unpatched code out of hand.
As far as performance goes, the unpatched code took 900 seconds on 1 month, my patched code took 41 seconds on the same month, Tony's took 81 seconds. Using 6 months of production data, my patch took 258 seconds and Tony's took 447 seconds. I did not attempt the unpatched code over 6 months, it was obviously going to be much worse.
Lastly, I examined Tony's patches and I have an additional patch for them (https://github.com/tbannister/moodle/commit/d0f4422e329b07986050adafd8d7989c55ba8398) that produces output identical to the original statslib code.
Okay, I think I understand all of the issues now...
So, Tony's patch rewrote the queries that were problematic in a more correct way. Tyler's patch substituted temporary tables for corrected queries. If this is the case, then it does make sense to recommend Tony's patch.
Now I understand that further, Tony may have modified the queries to correct for potential functional issues with the old system. If this is the case, then I think those corrections should be done in another issue, specifically dealing with those bugs. That way, the performance improvement can be verified easily by checking that the same data is output as before the improvement.
I believe Tyler has proposed that above, and supplied a modified version of Tony's patch, changes to provide the same output (right or wrong) as before the improvement. That should be the one we recommend for this issue.
Now, if there are other issues with the function (bugs?), let's create a new issue to deal with them and use Tony's patch with the bug fixes as a candidate for that fix.
Make sense?
I'm still not sure that replacing incorrect stats calculations with a faster version that still generates incorrect stats is quite right - especially if those incorrect stats were found as part of the patch. This sounds like the sort of thing that would get rejected by the integrators to me.
Also - MD has given his +1 to accept a patch that:
- adds the new field with the read/write type
- change logging code to make sure that the field is filled during upgrade, and always kept full after that whenever entries are added (possibly slow) or at least on a cron job.
neither of the patches here do this - Petr has voiced his concerns over the use of temp tables too. (this doesn't necessarily mean the integrators or MD will reject a patch like the one Tony/Tyler have been working on) - their patches are also useful for others wanting to apply a fix to their local installs to speed up stats.
Neither of the patches do that because there isn't any need and it makes little sense! That part of the calculation is very fast - it will just serve to further bloat the mdl_log.
Petr's temptable concerns are around mysql not supporting multiple references; part of the rewrite eliminates all multiple-reference cases.
Tyler, can you give any details about the different rows? Maybe a copy of them? "X different rows" doesn't help track down any problem.
Again, it is really bad for upstream to have a set of queries that does not work reasonably in all possible cases. That is exactly why this issue exists in the first place.
yep - no argument from me here - I just wanted to make it clear to Mike that Martin hasn't actually given his +1 on this particular design/patch. nice to Hear Petr's concerns are less of an issue with the latest code too.
Dan - Ultimately, replacing incorrect stats would be a goal, yes. But in a development process, its usually best to tackle one problem at a time so that testing and verification can be made easier. In this case, this issue was created to deal with performance issues. I believe we have come to an agreement on the patch that should be used.
Now, there was extra changes made in Tony's patch to fix another problem. If we include that with this issue, and testing fails, we will have a difficult time determining whether it was the performance changes or the functional changes. That's why I prefer to separate them. If we can validate the performance changes as being correct, it will be easier to validate the functional changes.
Next, we don't really have a problem statement for the functional changes that were introduced. We need to know what they are designed to fix, and what the test cases are to verify them.
I believe we should go ahead with the performance patch, recommending it for release. No guarantees it will be accepted, but in the absence of a full rewrite, it is a better compromise than doing nothing.
I've done some additional testing using some new data with 100,000 users. Using this heavier load, my 4 table patch took 15,402s to generate a years worth of daily logs, while my 1 table patch (based on Tony's patch) took 52,039s. On the larger load the 4 table patch was more than 3 times faster.
I discovered a small performance problem with temporary tables when they are repeatedly created and dropped. Running my test on two years worth of data showed a significant increase in the time it took to create new temporary tables. Accordingly, I've moved the table creation and dropping code out of the daily stats loop, and replaced that drop/create part with "truncate" queries instead. I also added some additional output for weekly and monthly stat generation.
Tony, to check for differences, I'll have to run the test again with your code. So far I've only been looking at performance with this new, larger, set of data.
Interesting, is that persistent across fresh connections, or are you running the 2 years in a single go?
If it only happens when doing 2 years in a single cron script, I would say that isn't really worth worrying about too much; nice quirk of mysql though.
It only occurs when running 2 years in a single go. Obviously, it's not going to be a common occurrence, however, it's actually less work to truncate the tables instead of creating and dropping the tables in the loop. I figure we should probably do it the right way since it also fixes an unusual case at the same time.
The updated code is available on github
Hi Tyler, Tony, Dan, Mike and all others involved here.
Michael assigned this this issue to me so that I can help out or complete things to make sure we don't miss out on the excellent discussion and coding that has gone on here.
I've been looking over this issue and your patch Tyler at https://github.com/tbannister/moodle/compare/master...MDL-30643_master-log-speed for the past day.
I like the look of the way things have shaped up. The use of temp tables appears really smart and seems well suited to the present situation.
While I agree with Petr's earlier comments about the log systems as well as stats needing a general overhaul I think certainly that if we get this code tidied up a little and verify its effects across the 4 DB's we support in core then it should land.
As mentioned I've already been looking over this code, and have talked to Eloy about this issue and the proposed solution as he really is our database expert.
As I see it at the moment there are three things that need to happen before we can get this issue up for integration and eventually into core.
1. Review and tidy up the code.
There is one large change I believe needs to be made that came from discussions with Eloy. We think it would be best to create temp tables using a method similar to that used when creating the temp backup tables.
Rather than construct the temp tables in code as is happening now within stats_temp_table_create we think it is best those temporary tables are created from templates.
In the case of backup that is handled through backup_controller::create_temptable_from_real_table. It then creates temp tables from the real tables that have been set up as templates e.g. backup_ids_template and backup_files_template.
Taking this approach may not be quite so quick but having those tables defined in a standard way helps make them more visible and manageable in the future.
In this case things could also be done a little differently. The stats_daily and stats_user_daily tables are nearly identical to the temp tables you are creating. If we get indexes sorted then we could easily use those as the templates for their temp table equivalents.
The temp log tables are a different story, really they would require a separate template table. Perhaps called stats_log_template.
Other than that I havn't spotted anything requiring significant changes. Naming of tables and functions is perhaps the only other thing worth mentioning. Naming the temp tables with "_temp" as a prefix worth doing and a more consistent scheme to temp tables already being used in Moodle.
2. Guarantee results through unit tests with a crafted dataset.
In discussing this with Eloy we also felt this would be something that would be not only essential to getting this integrated but also highly beneficial in ensuring that things were being generated with the same accuracy that they were prior to this patch. As someone mentioned above this issue should focus on performance. Any efforts to improve stats generation should be undertaken in another issue.
With unit tests we would have an easy means of testing any and all changes, both as part of this issue and in any future issues concerning stats generation.
Certainly it will help prove any changes being made in queries.
3. Ensure operation and performance on all supported databases.
This is really just about testing. Certainly having the unit tests above will help us in this, but definitely it is still worthwhile running the stats generation on large datasets in all 4 databases and monitoring performance to ensure we only move performance forwards and not backwards in all systems.
This should also be reflected in the test instruction for this issue when it does get integrated so perhaps creating a means of generating the required data would help.
Really the first question I have is for Tony, whether you would like continue developing this solution or whether you would like me to take over and finish off development, namely the tasks noted above?
If you would like to continue development then I will peer-review the code you've got properly, give detailed feedback and help you out in any other way I can.
If I take over development then I will review and make changes as I go, of course I will make any changes on top of existing commits kudo's aren't lost.
I'm quite happy either way.
I've begun work on the review and code tidy up. I will note that there are some caveats on the create_temptable_from_real_table function, it leaks memory and increases memory usage by approximately 10 MB, but I have none the less incorporated it's xml loading procedure. I will run a test on my big data set soon to see how large the performance impact is.
If I understand correctly, the temp log tables should be in lib/db/stats_log_template.xml?
I'm using "temp_" as an additional prefix, this will make the table names (for the standard prefix mdl_) mdl_temp_log1, mdl_temp_log2, mdl_temp_stats_daily and mdl_temp_stats_user_daily. Were you saying that you wanted the tables names to be _temp_log1 or is mdl_temp_log1 acceptable?
I'm beginning work on test cases now, and I think that may turn out to be the majority of the additional work required.
The updated code with test cases is now on Github.
I changed the file name to temp_stats_log_template.xml to emphasize that it was for temp tables, and I used the <prefix>temp_<table> format for the table names. The test cases took quite a bit longer than the actual fixes, but the test file does test all of the daily stats related functions. All of the db queries will be triggered by the test cases. The should be a problem running the test cases against unmodified code because I changed the name of one of the functions (stats_daily_progress to stats_progress because it's now used to print progress for weekly and monthly stats) so two test cases (test_statslib_progress_debug, and test_statslib_progress_no_debug) won't work. Lastly, I did not write any tests for weekly or monthly stats.
Hi Tyler,
thanks for your hard working! I'm wondering if someone, when the issue will be closed for 2.x, will also take care to backport such work in 1.9 (Ref.:
MDL-18484) even if 1.9 is "just" under security fixes maintenance.
my -1 for a 1.9 backport - it would require significant testing across multiple db types which I'm not too keen on doing myself.
1.9 is only really supported by us for Serious security issues - if this fix encourages people to upgrade to 2.x that's a good thing!
I found a small discrepancy between my code and the original statslib code, when certain logs weren't present a blank daily stats record wasn't produced in the new code that was produced in the original. I've fixed the new code to produce that record and updated the test cases to reflect that those log entries should be present. So Tasks 1 and 2 should be complete now.
Taking this out of the Stable Sprint so that it can be worked on when the Sprint version is archived.
Well, I backported the code anyway since we're going to apply the fix to our customer's M19 sites. The backported code has been attached to one of my comments in
The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.
TIA and ciao
Ok, I've rebased the
MDL-30643_master-log-speed and master branches to today's release.
Hi All,
I've picked this up for review pretty much expecting this to be able to land soon given everyone's dedicated involvement here. However,
looks like we need some minor touch up (to
MDL-30643_master-log-speed branch) here before touch down:
- there are 4 calls to get_context_instance() which need to call either context_course::instance() or context_system::instance() as the former is deprecated now for master.
- i've attached a log (
MDL-30643_pu.log) of my phpunit test failures, i'm not sure why they're failing for me. (php 5.4.5 here). will look at this in a few hours.
- suggestion : the statistics cron runs fine for me but on hardly any data, it would be cool to have some generated dataset with variety to test performance here or perhaps some data with benchmarks for comparison against this patch in the test instructions. (maybe some comment in the test code about the expected speed for some given system just as a scale to compare against)
Hi Aparup,
- I've removed the references to get_context_instance() and I realized that I had not put the final changes into the MDL-30643_master-log-speed branch. The final change adds another temporary table that eliminates the repeated joins between mdl_enrol and mdl_user_enrolments. It mirrors a similar change I made to the
MDL-18484patch when back porting the fixes to Moodle 1.9.
- It looks like there were some errors in the test case handling of time zones. I think I've got the test case issues fixed.
- I have a dataset that I tested on, however, it's about 26 GB uncompressed and takes about 7 days to load into a distribution standard MySQL server. I don't think I can get it down to a manageable size.
thanks for that
that looks good now and has been integrated into master for further testing..
please feel free to run any tests against integration code with any data available to you.
Hi, FYI I'm getting 10 phpunit test failures with your patch applied against postgresql (seems to pass ok against mysql):
(I also get some other temptable errors at the end of the execution, perhaps you are missing to drop them after use or so?)
So, if you can investigate that a bit...TIA!
hi, the "protected function prepare_db($logs) {" is wrong, tests should not insert data directly into system tables - we have data generator for that...
To integrators: next time please make sure integrated code uses our SQL indentation style, thanks.
Test result: Unsuccessful.
This ran fine on MySQL, but I ran into the following errors when I ran cron with other DBs...
Running daily statistics gathering, starting at 1334073600:
Temporary tables created
Enrolments caclulated
!!! Error writing to database !!!
!! SQLState: 23000<br />
Error Code: 8101<br />
Message: [Microsoft][SQL Server Native Client 10.0][SQL Server]An explicit value for the identity column in table '#mdl_temp_log1' can only be specified when a column list is used and IDENTITY_INSERT is ON.<br />
INSERT INTO #mdl_temp_log1
SELECT id, userid, course, action FROM mdl_log l
WHERE l.time >= 1334073600 AND l.time < 1334160000
[array (
Error code: dmlwriteexception !!
!! Stack trace: * line 410 of \lib\dml\moodle_database.php: dml_write_exception thrown
* line 258 of \lib\dml\sqlsrv_native_moodle_database.php: call to moodle_database->query_end()
* line 365 of \lib\dml\sqlsrv_native_moodle_database.php: call to sqlsrv_native_moodle_database->query_end()
* line 740 of \lib\dml\sqlsrv_native_moodle_database.php: call to sqlsrv_native_moodle_database->do_query()
* line 1608 of \lib\statslib.php: call to sqlsrv_native_moodle_database->execute()
* line 194 of \lib\statslib.php: call to stats_temp_table_fill()
* line 451 of \lib\cronlib.php: call to stats_cron_daily()
* line 88 of \admin\cron.php: call to cron_run()
- PostgreSQL
Running daily statistics gathering, starting at 1332864000:
Temporary tables created
Enrolments caclulated
!!! Error writing to database !!!
!! ERROR: duplicate key value violates unique constraint "mdl_templog1_id_pk"
DETAIL: Key (id)=(1) already exists.
INSERT INTO mdl_temp_log1 (userid,course,action) VALUES($1,$2,$3) RETURNING id
[array (
'userid' => 0,
'course' => '1',
'action' => '',
Error code: dmlwriteexception !!
!! Stack trace: * line 410 of \lib\dml\moodle_database.php: dml_write_exception thrown
* line 239 of \lib\dml\pgsql_native_moodle_database.php: call to moodle_database->query_end()
* line 819 of \lib\dml\pgsql_native_moodle_database.php: call to pgsql_native_moodle_database->query_end()
* line 206 of \lib\statslib.php: call to pgsql_native_moodle_database->insert_record_raw()
* line 451 of \lib\cronlib.php: call to stats_cron_daily()
* line 88 of \admin\cron.php: call to cron_run()
- Oracle
Running daily statistics gathering, starting at 1334073600:
Temporary tables created
Enrolments caclulated
!!! Error reading from database !!!
!! ORA-00001: unique constraint (MOODLEUSER.M_TEMPLOG1_ID_PK) violated
Error code: dmlreadexception !!
!! Stack trace: * line 407 of \lib\dml\moodle_database.php: dml_read_exception thrown
* line 274 of \lib\dml\oci_native_moodle_database.php: call to moodle_database->query_end()
* line 1208 of \lib\dml\oci_native_moodle_database.php: call to oci_native_moodle_database->query_end()
* line 206 of \lib\statslib.php: call to oci_native_moodle_database->insert_record_raw()
* line 451 of \lib\cronlib.php: call to stats_cron_daily()
* line 88 of \admin\cron.php: call to cron_run()
looks like this is a candidate for reversion before rolling..
- protected function prepare_db($logs) is indeed writing to system tables . an example for datagenerator is at course/lib/courselib_test.php test_reorder_sections().
- the failures for cron noted too on every DB except mysql (which is probably not reporting it). Thanks for testing Michael.
ps: http://docs.moodle.org/dev/Writing_PHPUnit_tests could use some elaboration on data generator.
EDIT: and Petr's reference to sql style has this upcoming doc for SQL @ http://tracker.moodle.org/browse/MDLSITE-1914
Reverted 8 (from tyler) + 1 (from Apu) commits. This needs a bit more of love before landing. Reopening, thanks!
I've rewritten the test cases to use the data generators. That change should, I think, fix one of the PostgresSQL test failures (casting "varying" to "bigint").
The MS SQL error appears to be easily solved by dropping the id column and specifying the values, when the data log if filled. Apparently MS SQL doesn't normally allow primary keys to be specified.
From most of the test errors and the error output posted by Michael above, it looks like there may be an error in insert_record_raw for PostgreSQL and Oracle. Under MySQL insert_record_raw generates a new unique key for the id column, but it doesn't appear to do so in PostgreSQL and Oracle. I don't know why the behavior is different.
In any case, I'll change the call on 206 to insert_record and see if that fixes the problem. I should updated code ready tomorrow morning.
insert_record_raw() is a low level method, it does not do any data validation, please always use only insert_record().
Moving this reopened issue out from current integration. Please, re-submit it for integration once ready.
Just to be clear, the problem wasn't validation, it was failure to generate a suitable primary key id. In the MySQL implementation of insert_record_raw, a valid primary key id will be generated. If an id is passed, the MySQL insert_record_raw will drop the id and generate a new one. In this case, no id was passed and an already used primary key id was generated for PostgreSQL and Oracle. The code worked in MySQL because the insert_record_raw did the right thing for MySQL and it failed in PostgreSQL and Oracle because insert_record_raw did the wrong thing for those databases.
But yes, it should have been insert_record() instead.
1/ Oh, the problem may be somewhere else - you can not just insert records with fixed 'id' into our tables in function stats_temp_table_fill(), you MUST reset table sequences afterwards using $database_manager->reset_sequence($tablename).
2/ the $DB->insert_record_raw('temp_log1', array('userid' => 0, 'course' => SITEID, 'action' => '')); expects normalised values for given database - in this particular case action value '' would be converted to NULL in Oracle which is wrong.
Thanks Petr, I didn't realize empty strings would become NULLs in Oracle.
I've now updated my
MDL-30643_master-log-speed branch with fixes for all of the testing issues mentioned above. I've setup a local PostgreSQL DB and tested the updated code locally on both PostgreSQL and MySQL and it's passing all the tests for both.
I don't think I can get around having to insert the log entries directly into the database (using insert_record now) because I need to create log events that occurred in the past, and add_to_log() always uses the current time.
Thanks Tyler for working on this issue and especially for listening to feedback!
When is the next opportunity to get this issue back into integration and testing?
The issues identified in testing should be fixed now. The changes have been tested in MySQL and PostgreSQL.
Tyler: integration starts weekly on mondays and testing is weekly on wednesdays (barring special release weeks/freeze etc)
Thanks Agarup, I assume requesting a peer review was the proper way to get that process rolling.
Ok, this has been waiting for review for more than a month now. Is there anything we can do to encourage the review?
The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.
TIA and ciao
The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.
TIA and ciao
Ok, I've rebased the branch again. Is there any chance we can get the integration review done before the next weekly update?
Hi Tyler,
I've just looked at the commits here and they have commits mentioning 'Revert "Revert ...."' . In the case of reverted issue/commits, the same rebasing strategy should be applied instead re-reverting the commits. (i've clarified this with a note here just now http://docs.moodle.org/dev/Integration_Review#Schedule)
For now i've picked and reworded the commits here : https://github.com/nebgor/moodle/compare/int_master...int_master_MDL-30643
Hi Tyler,
I've gone through this now (with Petr who was in HQ for the day actually) and here are my notes :
- As Sam mentioned in his comment, a function similar to backup_controller::create_temptable_from_real_table seems the best way to go but this seems to work atm so i'm asking Eloy about his opinion.
- TRUNCATE is not really safe to use here perhaps look at using $DB->delete_records($name) there which does the job too.
- https://github.com/nebgor/moodle/compare/int_master...int_master_MDL-30643#L1R1579 stats_temp_table_fill($timestart, $timeend) has code which seems sql-injectable , perhaps ensure that they're integer.
- there are print_r() calls within the phpunit tests as well as trigger_error() calls. phpunit tests will be run from the cli and these do not apply here well at all. if you're looking to report a failure here, i'd suggest using an assert statement else i'd remove them.
- Also see the trigger error here although i'm not sure if asserts are fine in the setup stage.
- the large arrays of test data can be from loaded from csv/xml type files which would make this lots more readable (in a separate test data file) : see lib/phpunit/classes/advanced_testcase.php for createXMLDataSet(), createCsvDataSet() and friends there. (available as part of advanced_testcase).
- test_statslib_get_action_names() has lots of hardcoded numbers,
- if (!$DB->execute()) type calls aren't going to bode well here as the call only ever returns true and throws dml_exception otherwise. I'd just let it throw an exception as this sort of failure would mean phpunit isn't going anywhere later.
- test_statslib_progress_no_debug() looks like it can easily be a false positive in future after code or simple string changes. i'm not sure how to better test this atm.
- test_statslib_get_action_names() has the hard coded actions, perhaps these could be constants in stats lib or somewhere in logs ? not sure.
- there are minor inconsistencies with the coding style with regards to keywords such as 'COUNT' and 'AS' etc within statslib.php that could be made more consistent (or according to what might be evolving @
overall i think this should go in once its cleaned up as it definitely seems to be helpful
imo, the lib/db/temp_stats_log_template.xml file seems to be starting an undocumented convention whereas as suggested this could be created from an existing table defined within install.xml. (again, i'll ping Eloy here about this)
I hope thats helpful feedback Tyler, looking to see this land.
ps: Another point is that if this is really useful, then we have a need for a proper temp-table usage api within some DB layer API. this could possibly work similar to backup_controller::create_temptable_from_real_table().
pps: rebase and get rid of those recursive revert messages
Aparup, I'm afraid your comment on the reverts isn't clearing things up for me very much. I originally reverted the revert of my commits because they would no longer apply cleanly onto the master branch (I think, because they were reverted). Only my newest commits were applied. If I understand correctly, you're saying that instead of reverting them I should cherry-pick (or otherwise add) the commits back onto the tail end of the master branch? Rebasing was a disaster when I tried it. Instead of reverting the commits and editing the message so that the REVERT "REVERT " part of the message is removed.
On the bullet points:
- That might be a good basis for an improvement issue.
- Ok
- You are correct that it would be SQL-injectable (I'll fix that) but it was only ever called with internally generated values.
- I think only one of the print_r codes is reachable in the code, the others are commented out (and will be deleted). I can remove it, but it's there to help debug the test case in case it fails to setup properly. I don't think the assert can be called safely in any of the cases where trigger_error is called they are all pre-test case code.
- I don't think they are. The other print_errors are also in setup code.
- Ok
- I assume you mean strings (the numbers are irrelevant). I don't know of a better way to generate those lists.
- Ok. I forgot to remove the if statements when I ported that from my MOODLE_19_STABLE stats code.
- I could make the string that's print for nodebug a defined value (STATSLIB_PLACEHOLDER_OUTPUT or something like that).
- I think adding defined names for those actions falls outside of my scope here (since the actions names are defined in the logging code).
- I might as well fix up the COUNTs and ASs. I assume you mean they should all be all-caps?
Hi Tyler,
Yup cherry-picking would do too for commits that don't change. any effort to achieve the rebase concept really. I believe rebasing fails usually with larger strings of commits where not every commit is changed. Hm, perhaps the rebase message can include some hint towards handling of reverts. (this has messed up a few devs here at HQ and i'm hoping to make this potential time waster go away)
On the points (avoiding bullets :-D):
- yes agreed, it seems theres more thought needed here bbut for now (after discussing with Eloy), to create less overhead for changes in future, seems better to stick with what exists and afford the 2-3 secs needed for having the tables defined as standard i.e.: having the table definition inside install.xml and creating a copy for temp usage. I'll create an issue about making this more standardised/normalised (
- cool, Altitude 1000ft
- Just staying safe, Altitude 900ft
- I'm not able to understand why the setup would fail code-wise. Other unit tests should pick up DB failures otherwise its infrastructure setup failing. In any case, imo this failure would be beyond the scope of the unit test. If debugging() is required, perhaps they should be within the code being tested rather than within the test itself.
- cool. I think there is scope for this in another phpunit improvement issue. (
- Altitude 800ft.
- oops - i hadn't finished that one and can't recall but i think some 'future proofing against development changes' direction was there. don't think its any game stopper.
- Altitude 700ft.
- agreed. Altitude 600ft.
- yup out of scope, (MDL-36416)
- yes. (hopefully this changes a few commits to allow an easy rebase instead of picking)
(/me turns off MSAW for http://www.youtube.com/watch?v=v_z5HtME9n8)
ping! any changes? this is seeming like it won't make for 2.4 atm.
I'm reopening this for now. (unfortunately point 1 is the sticking point)
Please resubmit when ready.
(ping for faster response in devchat if necessary for 2.4)
Ok, just to be clean for #1 you want me to remove the xml file for the temporary database tables and simply add them into the primary install.xml file?
I'm working on #6 right now. I takes a while to work out how all of this is going to work together, I could not find any examples in Core where an XML file with placeholder values has to be loaded and parsed and then compared against the contents of the database.
I think everything except #1 and #6 is complete now.
Also, Aparup, could you make sure that test case issues outlined here are documented somewhere, it's a bit frustrating to have to keep rewriting test cases to meet what seems like an evolving standard. For instance the Writing PHPUnit tests page does not mention using XML/CSV files to load data at all.
As per Eloy's comment in dev chat, I've renamed the temp tables in install.xml to use the name syntax of "temp_<table>_template". That should avoid any possible problems with name conflicts. The updated code has been pushed to Github.
Cool, thanks Tyler.
(ftr:dev chat @ https://moodle.org/local/chatlogs/index.php?conversationid=11443)
I've added http://docs.moodle.org/dev/Writing_PHPUnit_tests#Large_test_data which hopefully improves visibility of the short explanations as well as tagged
MDL-32960 as dev_docs_required (for stub expansion).
Hi Tyler,
i've just come across one bug (minor) in the unit test:
1) statslib_daily_testcase::test_statslib_get_start_from
Log entry start
Failed asserting that '1272657610' matches expected 1272700810.
To re-run:
phpunit statslib_daily_testcase lib/tests/statslib_test.php
so i've added one commit and tested across different times and it seems fine.
mysql and postgres looking good, i'm just testing now on mssql and oracle before integrating this.
oh i don't have a working phpunit on windows and seeing that
MDLQA-5180 MDLQA-5181 MDLQA-5182 and MDLQA-5183 haven't been done yet atm i'm pushing this up now.
this has been integrated into master only.
ah the integration checks have sprung up missing upgrade steps for the tables.
Table temp_enroled_template only available in first DB
Table temp_log_template only available in first DB
i've now added the upgrade script into integration.git (7d27055)
I'm attaching phpunit test results for oracle and mssql (on windows) i've received from Michael here. The failures are 9 on both.
Log entry start
Failed asserting that '1272592810' matches expected 1272657610.
was thrown up by integration servers whilst it works for me here..
also noting another :
Failed asserting that 0 matches expected 1.
It looks like all of the stats generation tests are failing under MSSQL and Oracle, and the change I made to prevent problems with stats generation from disabling the cron (handling the exceptions) is preventing phpUnit from reporting the actual problem.
Hi Tyler,
regarding the self::DAY timestamp, our friend is get_timezone_offset() in stats_get_base_daily().
So i've realised we cannot use $CFG->timezone = 99 in any unit tests. i've pushed up a fix to set it to 8.
regarding mssql/oracle turbulence, after setting $CFG->timezone = 8 i can get these 9 failures on postgres (on integration server). I'm attaching a file now which has a bit more data about the errors. from a glance they all seems to be related to time or timezones as the expected data is off by some similar amounts (eg: +-63200).
What's strange is why i can't replicate it locally... hence timezone guess again.
Hopefully we can resolve the mysterious 9 now
could it be that adjusting the self::DAY value has resulted in these discrepancies? if so then feel free to propose a patch against integration that changes the self::DAY back to 1272592810 together with what the $CFG->timezone should be. goodnite.
I found a few problems in the test code. It has to do with the wonky way that day starts are calculated, and a call to date that could potentially make the code not run if the configured timezone didn't match the machine time zone. It would abort the starts because the configured (system) time was too far away from current (Moodle) time.
I think it's fixed now, I tested it with a number of different time zone settings and it always seems to work now. I've added a commit that fixes the problems to the pull branch.
Thanks Tyler,
I'm running these now in a few timezones as well atm. Wonky is the word! But the performance gain is what we want in this issue and the test should work to have a comparison against, so i think we're doing fine so far.
pg and mysql are passing but i'm hearing from MdR that mssql and Oracle still aren't. i'm wondering if there are timezone specifics at play here for these DBs (i'm not aware of any)
I've attached the PHPUnit test results for Oracle and MSSQL after the most recent changes.
i've got a freetds installation that is thoring these up as well on which i can throw more debugging messages on hopefully soon..
Aparup, the easiest way to get more information is to remove the output buffering calls. The output buffering is hiding the mtrace messages that would normally be output by the cron.
Ok, I've pushed a new update that works around the output buffering to print the mtrace output if the test fails to complete. Also, if the Moodle site has debugging enabled (at least DEBUG_ALL) the SQL errors will be in that mtrace output. Hopefully this will help us get some useful output from the MSSQL and Oracle tests.
I've attached the output after picking the debugging switch patch (12ccfc1).
i'm not sure what quirks are up to no good now and temp tables are proving difficult to debug with.
Petr, Eloy,
could you look at phpunit_
MDL-30643_debug_n.txt attached and lend some DB expertise here
i should probably add the extract from the file which is :
0:0 Error writing to database (Invalid column name 'courseid'.
INSERT INTO #u_temp_stats_user_daily
(stattype, timeend, courseid, userid, statsreads)
SELECT 'logins', 1272758400 AS timeend, 1 AS courseid,
userid, COUNT(id) AS statsreads
FROM #u_temp_log1 l
WHERE action = 'login'
GROUP BY timeend, courseid, userid
HAVING COUNT(id) > 0
[array (
...error occurred, completed 0 days of statistics in 0 s.
Just had a quick look at this, don't have much input, apart from I noticed {{ $tempfile = $CFG->dirroot . '/lib/db/temp_stats_log_template.xml';}} is unused??
I've attached new PHPUnit output with the extra debugging output for MSSQL and Oracle.
Thanks Micael,
interestingly the ORA-00904 has more info up and about
some further notes that we can clean up here,
- Dan's comment
- statslib.php stats_temp_table_create() ddlunknowntable ddl_exception uses undefined $name
ok i've replicated the query problem with :
INSERT INTO [moodle_gitintegrationmaster].[dbo].[mdl_stats_user_daily]
(stattype, timeend, courseid, userid, statsreads)
SELECT 'logins' as statype, 1272758400 AS timeend, 1 AS courseid,
userid, COUNT(id) AS statsreads
FROM [moodle_gitintegrationmaster].[dbo].[u_temp_log_template] tlt
WHERE action = 'login'
GROUP BY timened, courseid, userid
HAVING COUNT(id) >0
and its led me to believe that this might work (i don't know why we're grouping by a fixed number in this query s to the server so i've removed it):
INSERT INTO [moodle_gitintegrationmaster].[dbo].[mdl_stats_user_daily]
(stattype, timeend, courseid, userid, statsreads)
SELECT 'logins' as statype, 1272758400 AS timeend, 1 AS courseid,
userid, COUNT(id) AS statsreads
FROM [moodle_gitintegrationmaster].[dbo].[u_temp_log_template] tlt
WHERE action = 'login'
GROUP BY userid
HAVING COUNT(id) >0
i've updated that commit to fix all other similar SQLs. please check them.
I've also put in a separate wip commit which i'm really actualyl not commited to but with it its looking like the fails are reduced to 7 and the data is whats failing. So no SQL woes. hopefully we can get to a stage where the tests are passing Tyler
Aparup, I incorporated your changes into my pull branch. To fix the data issue, I changed the final table to work differently, I removed some of the hard coded outer query values and instead return the required values from the inner query and group by the returned columns. I think that should allow that query to function in MSSQL and Oracle while still returning the proper values.
Thanks for that Tyler. I had created my changes in a separate commit for you to be able to pull it in (saving time) instead of having to create another commit
Anyway, i've now tested this with all our latest changes and its passing for me for my mssql (freetds) setup. I'll ask Michael to run phpunit on oracle and sqlsrv once more to verify.
Test result: Success.
This now seems to work across DBs. I ran the related unit tests and also ran cron to run the daily statistics. I can't testify to an improvement in speed, but I hope partners will be able to let us know.
Kudos to all of you! Borrowing the Animated Amy from Mary Evans:
I've added pull branches and diff urls for 2.2 and 2.3. I didn't include test cases in the 2.2 branch since they won't work without the changes in 2.3.
Removing the 2.2 pull branch and diff url. It looks like disposing of temporary tables doesn't work properly in 2.2. When running the cron the following errors showed up on my test site (after completing the statistics generation):
Notice: Trying to get property of non-object in /home/tyler/projects/moodle22-plain/lib/moodlelib.php on line 6346
Call Stack:
11.2980 96343048 1. moodle_database->__destruct() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:0
11.2980 96343048 2. mysqli_native_moodle_database->dispose() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:140
11.2980 96343048 3. moodle_database->dispose() /home/tyler/projects/moodle22-plain/lib/dml/mysqli_native_moodle_database.php:414
11.2980 96343048 4. moodle_temptables->dispose() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:322
11.2981 96348256 5. database_manager->drop_temp_table() /home/tyler/projects/moodle22-plain/lib/dml/moodle_temptables.php:130
11.3063 96451760 6. database_manager->execute_sql_arr() /home/tyler/projects/moodle22-plain/lib/ddl/database_manager.php:522
11.3063 96452192 7. database_manager->execute_sql() /home/tyler/projects/moodle22-plain/lib/ddl/database_manager.php:75
11.3063 96452192 8. mysqli_native_moodle_database->change_database_structure() /home/tyler/projects/moodle22-plain/lib/ddl/database_manager.php:88
11.3065 96337592 9. moodle_database->query_end() /home/tyler/projects/moodle22-plain/lib/dml/mysqli_native_moodle_database.php:749
11.3066 96361720 10. ddl_change_structure_exception->__construct() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:400
11.3066 96362152 11. ddl_exception->__construct() /home/tyler/projects/moodle22-plain/lib/ddllib.php:130
11.3066 96362424 12. moodle_exception->__construct() /home/tyler/projects/moodle22-plain/lib/ddllib.php:81
11.3066 96362640 13. core_string_manager->string_exists() /home/tyler/projects/moodle22-plain/lib/setuplib.php:120
11.3067 96362984 14. core_string_manager->load_component_strings() /home/tyler/projects/moodle22-plain/lib/moodlelib.php:6444
Fatal error: Uncaught exception 'ddl_change_structure_exception' with message 'error/ddlexecuteerror' in /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php on line 400
ddl_change_structure_exception: error/ddlexecuteerror in /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php on line 400
Call Stack:
11.2980 96343048 1. moodle_database->__destruct() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:0
11.2980 96343048 2. mysqli_native_moodle_database->dispose() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:140
11.2980 96343048 3. moodle_database->dispose() /home/tyler/projects/moodle22-plain/lib/dml/mysqli_native_moodle_database.php:414
11.2980 96343048 4. moodle_temptables->dispose() /home/tyler/projects/moodle22-plain/lib/dml/moodle_database.php:322
11.2981 96348256 5. database_manager->drop_temp_table() /home/tyler/projects/moodle22-plain/lib/dml/moodle_temptables.php:130
11.3063 96451760 6. database_manager->execute_sql_arr() /home/tyler/projects/moodle22-plain/lib/ddl/database_manager.php:522
11.3063 96452192 7. database_manager->execute_sql() /home/tyler/projects/moodle22-plain/lib/ddl/database_manager.php:75
11.3063 96452192 8. mysqli_native_moodle_database->change_database_structure() /home/tyler/projects/moodle22-plain/lib/ddl/database_manager.php:88
11.3065 96337592 9. moodle_database->query_end() /home/tyler/projects/moodle22-plain/lib/dml/mysqli_native_moodle_database.php:749
Ok, found and fixed the 2.2 issue. Moodle 2.3 requires drop_table be used instead of drop_temp_table. I fixed the M22 branch to continue using drop_temp_table and the errors are no longer triggered.
Many, many thanks for your effort!
Millions of people will enjoy the results of your work, yay!
Closing as fixed. Ciao
Does anybody know if these changes be incorporated into M22 and M23?
Hi Tyler,
no, it was only integrated to master. So only it and now 24_STABLE have your awesome improvement available.
For sure it won't be backported to 22_STABLE ever (near becoming out of support) and 23_STABLE maybe a potential target, although right now I feel myself positioned against that.
If you think it would be worth backporting to 23_STABLE, plz, fill an issue about that to have a proper place to discuss. But, by default, it's against the integration "politics". More if there is already a stable version fixed and the previous one is not new-new (say 4 months or after .2 minor release).
Hi Eloy -
Can you explain why you are "positioned against that", and why "it's against the integration 'politics'"? The biggest problem we have for us is that many clients will remain on 2.2 for many more months, and this issue prevents them from using this feature.
Hi Mike,
i've been away for awhile so i'm only just reading this and here's what i can add.
The general policy for changes to STABLE branches is to have minimal developmental changes and majorly bug fixes. Slow queries don't necessarily fall under 'bug fixes', but if the improvements are still required, backporting can be suggested via another MDL. Backporting of improvements onto a stable branch carry more risk than development and improvement on master as the stable branch doesn't go through the entire QA process again after it has been released.
For 22, timing has made this very difficult to do safely. As Eloy has mentioned, 23 maybe considered in a separate MDL.
There is a random (time-depended?) stats unittest failure, I created an issue MDL-46773 about it


text:
vintage inclined
I find vintage shopping on the whole pretty tedious. I don't like sifting through rubbish to get to the good bits and on the rare occasions I find something I like it definitely won't be in my size. However thanks to the internet, It's gone from sifting through piles of musty clothes to being all styled up and photographed online, laid out nicely and sized up properly. Vintage Inclined kindly sent me this tshirt with amazing appliquéd detail which is awesome. It has totally retro power shoulders too so I feel like a 80's businesswoman but I'll rock it anyway!


text:
This is one of several postcards that I have that show some of the places where my ancestors lived. The image is of the train depot in Beaufort, Carteret, NC. The postcard reads on the back: Historic North Carolina Collection, Copyright 1900-1939, Archival Edition 1999.
Just my way of sharing with family, friends, and anyone who will listen, the things that I have learned about my ancestors with a little bit about myself and my own little family in the mix.
Wednesday, July 31, 2013
Wordless Wednesday: Train Depot --Beaufort, Carteret, NC
4 comments:
Awesome Sauce!I recently ID'd my 2nd gr-grandpa in a railroad photo...Bought the Census occupation as a Porter to life as did this photo!ReplyDeleteReplies
It's amazing how photos really help us envision how our ancestors once lived. Like you said, it brings things to life.Delete
Where can I find postcards like these? Cool!ReplyDeleteReplies
Check Ebay. I've found quite a few that way.Delete
Subscribe to: Post Comments (Atom)


text:
- Product Details
- Fabric Estimator
- Rewards
This is a light weight, fine, silk and cotton blended voile. Very soft and luxurious, great for dresses.
Color: Citron
Width: 52"
Content: 40% COTTON, 60% SILK
Country: IMPORTED
Learn about fabric careHOW TO CARE FOR SILK:Dry cleaning is recommended for all silks.Mood strongly recommends test-laundering a swatch, if you plan to wash your garment or home furnishings yourself.
PLEASE NOTE: The above numbers are estimations only. Please take into account garment size, lining and fabric width.


text:
Moderators: BrickSyd, Duerer, Elmagnifico, Kommander Ken
Brikchat wrote:<Guest6854>: screw you, theres an elder god out there with a colette body pillow
Natalya wrote:Fredde, you don't win the internet with this.
You ARE the Internet.
Manly Santa wrote:I'll gi' ya a Glasow Kiss mun!
Varsaavius wrote:As the size of the explosion increases, the amount of social situations it can't resolve approaches zero.
lawmaster wrote:Kill him again.Bragallot wrote:He's already dead.lawmaster wrote:Kill the guy I'm in cc with.
Users browsing this forum: Bing [Bot] and 7 guests


text:
Quality CareFind out why Mayo Clinic is the right place for your health care. Make an appointment.
Meet the StaffFind a directory of doctors and departments at all Mayo Clinic campuses. Visit now.
Research and Clinical TrialsSee how Mayo Clinic research and clinical trials advance the science of medicine and improve patient care. Explore now.
Visit Our SchoolsEducators at Mayo Clinic train tomorrow's leaders to deliver compassionate, high-value, safe patient care. Choose a degree.
Professional ServicesExplore Mayo Clinic's many resources and see jobs available for medical professionals. Get updates.
Give to Mayo ClinicHelp set a new world standard in care for people everywhere. Give now.
Mayo Clinic offers appointments in Arizona, Florida and Minnesota and at Mayo Clinic Health System locations.
Subscribe to Housecall
Our general interest e-newsletter keeps you up to date on a wide variety of health topics.
Signs and symptoms of pheochromocytomas often include:
Less common signs or symptoms may include:
These signs and symptoms often occur in brief spells of 15 to 20 minutes. Spells can happen several times a day or less often. Your blood pressure may be within the normal range or remain elevated between spells.
Spells may occur spontaneously or may be triggered by such factors as:
Foods high in tyramine, a substance that affects blood pressure, also can trigger a spell. Tyramine is common in foods that are fermented, aged, pickled, cured, overripe or spoiled. These foods may include:
Certain medications that can trigger a symptomatic spell include:
The signs and symptoms of pheochromocytoma can be caused by a number of different conditions. Therefore, it's important to get a prompt diagnosis.
Although high blood pressure is a primary sign of a pheochromocytoma, most people with high blood pressure don't have an adrenal tumor. Talk to your doctor if any of the following factors are relevant to you:
Mayo Clinic does not endorse companies or products. Advertising revenue supports our not-for-profit mission.
Check out these best-sellers and special offers on books and newsletters from Mayo Clinic.
Any use of this site constitutes your agreement to the Terms and Conditions and Privacy Policy linked below.
A single copy of these materials may be reprinted for noncommercial personal use only. "Mayo," "Mayo Clinic," "MayoClinic.org," "Mayo Clinic Healthy Living," and the triple-shield Mayo Clinic logo are trademarks of Mayo Foundation for Medical Education and Research.
We comply with the HONcode standard for trustworthy health information: verify here.


text:
Showing 1-24 of 544 items
- Sort by:
The 1904 General Store came from the 1904 World's Fair. We sell fudge, chocolate bars, candy, and retail gifts/furniture. Open Wednesday-Saturday 10 a.m. to 5 p.m. and Sunday, noon to 4 p.m.
Stop by for any of the specialty or create-your-own pizzas or calzones/strombolis. You can side that with a salad and fresh cold beer.
4204 Main Street Brewing Company - Alton
This restaurant is known for the many varieties of beers that complement the food. Brews include Blonde Ale, Chocolate Stout, Saison ( a farmhouse ale brewed in the Belgian style), several different IPA's. Hefeweizen and more. Menu includes certified angus beef steaks, pastas, seafood, burgers and sandwiches.
Acapulco Family Mexican Restaurant
Acapulco has a variety of authentic Mexican cuisine for the whole family.
CITY:Granite City
Aerie's Lodging & Terrace Conference Center
At Aerie's, the individual cottages include jaccuzzi tubs, kitchenettes and private decks. The Lodge, perfect for family gatherings, includes a common room, full kitchen and a deck shared by four en-suite rooms. The Vacation Villas offer multiple arrangements and include private deck space with access to the swimming pool and clubhouse. Free Wi-fi.
The view is incredible and the wine selection is unbeatable every season of the year on top of the bluffs at Aerie's Winery. During the spring and summer, the outdoor deck offers a breathtaking view of recreation on the river. As the temperatures cool, you can lounge inside next to the fireplace. Aerie's also offers cottages and a lodge to accommodate the needs of overnight guests, all within walking distance of the winery. Seasonally, check our calendar for live music on the decks Wine, cheese, sausages and pizza are available.
Alfonso's has delicious Italian food with excellent service and carry-out.
During your stay in the area come and lose your everyday stresses at Along the Way guesthouse. It offers alternative accommodations for travelers. It is your home away from home where you will experience gracious mid-western hospitality. Three suites available.
There is something for everyone at the Alton Antique Center. Merchandise from around the world, from inexpensive to investment quality pieces, abound in this shop located on the banks of the Mississippi River. Rare one-of-a-kind items, exotic pieces, oddities, and items from Asia, Great Britain, Europe and America are available. From beautiful to whimsical, there is always something to capture the eye and the imagination at the center.
Alton Cracker Factory Great Rivers Lodging
The Cracker Factory is nestled in the scenic Mississippi River town of Alton, Illinois. Guests can stay in the luxury and comfort of the Cracker Factory while visiting Alton for business or leisure. While there, they find easy access to many historic sites and the renowned Antique District. A variety of restaurants, entertainment venues, and cultural events are waiting to be enjoyed.
New merchandise arrives daily at this 13,000-square-foot store. From antiques to crafts to gifts and collectibles, there are unique items available to fit nearly every budget and taste. Over 100 independent dealers offer a wide variety of merchandise. Doors are open Monday through Saturday from 10 a.m. to 7 p.m. and Sunday from 12 to 5 p.m.
Based on the book Haunted Alton by Troy Taylor, Alton Hauntings Tours offers an entertaining and spine-tingling trip into the unknown, taking visitors to the most authentic haunted places in Alton. Uncover the eerie folklore, ghostly tales and documented haunted spots on a three-hour walking tour of the city's most haunted sites. Visitors will see sites that have become nationally known, like the old Alton penitentiary, First Unitarian Church, Enos Sanitarium and many others. The tours have been proven to be popular with ghost enthusiasts and history buffs alike, and nowhere else can you learn as much about the real, haunted history of Alton. Walking tours and bus tours are offered during two touring seasons – Spring & Summer and the Haunted Fall Season. Private tours are also available for groups of 20 or more. Reservations must be made in advance for all tours.
Looking for quality restoration of your antique furniture? Check out Alton Landing which offers a complete restoration service. Repair work is also available for pottery, clocks and chair caning.. Canvas boat covers are also available. Open Monday - Saturday from 10 a.m. - 5 p.m.
Entertaining audiences since 1934, the Alton Little Theater offers a season of great live theatrical performances from dramas to comedies to everyone's favorite musicals. The Alton Little Theater is one of the oldest community theaters in the state of Illinois.
A destination port on the Mississippi with picnic area's, playground, and outdoor pool. Within a half of a mile to Alton's historic antique district and the Argosy Casino.
Alton Museum of History and Art
Winged monsters, explorers, riverboats and a gentle giant. The Alton Museum of History & Art shows the crossroads of American history in Alton. The museum is located in the historic Loomis Hall across from the Wadlow statue. Loomis Hall is the oldest building in the state of Illinois continuously utilized for education. One of the most popular rooms, the Wadlow Room, pays tribute to Alton's "Gentle Giant" and the World's Tallest Man. The Pioneer Room explores the history of Alton from the Lewis & Clark Expedition to the Civil War with exhibits on Elijah Lovejoy, the Lincoln-Douglas Debates and the "Alton Route" on the Underground Railroad. Hours: Wednesday - Saturday 10 a.m. - 4 p.m., Sunday 1 - 4 p.m.
Alton Sports Tap is everyone's favorite place in town to catch a game. With screens all around, you are sure to find the best games on during every time of the year, and every seat in the house offers the perfect view. Kids and adults alike will enjoy our wide selection of pinball, darts and video games with a few classics including bags and golf. Pull up a chair and make a new friend where the sports are always on, the food is always hot and the good times are rolling!
Anchored by JCPenney and Macy's, Alton Square Mall is the area's regional mall destination. A full-service food court, U.S. Post Office, satellite city library and Christian Science Reading Room are also featured at the mall. Hours: Sunday, noon - 6 p.m. Monday - Saturday, 10 a.m. - 9 p.m.
Visit this working studio to shop for hand-crafted works of art - lamps, window panes, ornaments, sun catchers, glass beads and more. From decorative items to window panes, make a purchase from the floor or have Lynne custom design a piece for you and your home. Classes are also available. Hours: Wednesday - Saturday 10 a.m. - 9 p.m. or by appointment.
Historic Downtown Alton is better known as "Antique Row," boasting of more than 40 antique and specialty shops offering a wide array of treasures, trinkets, antiques and unusual gifts. Pick up a piece of Alton's rich history from one of the many local antique dealers. From the simple to the sublime, just about everything can be found along Broadway, Third Street and the surrounding neighborhood.
American Indian and Ethnographic Show
Historic artifacts on display.
83 room hotel that is handicap accessible, credit cards accepted, cable television, outdoor pool, pets accepted, restaurant within walking distance, complimentary breakfast.
This 71-room hotel has high speed internet access, an indoor pool, and is within walking distance to several restaurants.
55 rooms, non-smoking, handicapped accessible, credit cards accepted, cable television available, indoor pool, whirlpool, restaurant within walking distance, and a complimentary breakfast served.
CITY:Fairview Heights


text:
Beijing Olympics Get Big Green Tick
UNEP Report Spotlights Achievements and Highlights Some Shortcomings of 2008 Games
25th Session of UNEP's Governing Council/Global Ministerial Environment Forum 16-20 February
Nairobi, 18 February 2009 - The Beijing Olympics met if not exceeded many of its pledges on the environment, according to an assessment by the United Nations Environment Programme (UNEP).
From reducing air pollution to big investments in public transport and renewable energies, the organizers made major efforts to ensure that the 2008 Games marked a step forward in terms of an eco-friendly mass spectator sporting event.
These are some of the conclusions of Beijing 2008 Olympic Games - Final Environmental Assessment, which was released on 18 February during the UNEP Governing Council meeting.
The report underlines that more could have been done on engaging with Non-Governmental Organizations and cutting the Olympic and Paralympic Games' carbon footprint.
But overall the UNEP assessment, conducted by independently appointed assessors, concludes that Beijing raised the environmental bar and the Games left a lasting legacy for the city.
These achievements are all the more impressive given that the Games were held in a rapidly developing city in a country facing multiple development challenges in the first decade of the 21st century.
Achim Steiner, UN Under-Secretary-General and UNEP Executive Director who attended the summer Olympics, said: "The public attention of the world focused on the Beijing Games and whether the authorities could pull off a landmark event on many fronts including the environment. They have fulfilled the promise of a Green Games in many areas including public transport, waste treatment and green Olympic venues."
The challenge now facing Beijing is how to consolidate the achievements of the Games and turn them into a green legacy in the long term. UNEP's assessment of Beijing's achievements and challenges also provides key recommendations for the organizers of the Games in Vancouver, London and Sochi - as well as other mass sporting events - as they strive to realize their goals and set the environmental bar ever higher.
The Beijing Games and the environment
The Beijing organizers' goal was to offer a 'Green Olympics' to the world - overall, it is estimated that the authorities in China invested more than US$17 billion on environmental projects for the Games.
This plan was laid out in a series of twenty key environmental commitments ranging from the improvement of the city's transport infrastructure to the upgrading of the waste system and an increase in the city's green coverage. The city authorities also achieved a complete phase-out of hydrochlorofluorocarbons (HCFCs) - an ozone-damaging greenhouse gas -years ahead of the 2030 target.
Along with environmental improvements across the city, a powerful legacy of the Games is increased awareness of environmental issues in China, particularly among Beijing residents and businesses.
Conclusions and recommendations
The report highlights several areas where the Beijing Olympic organizers exceeded their commitments, including:
The authors also identified challenges in a few areas, including:
Air quality
Air quality was by far the most prominent environmental issue the Olympic organizers and the Beijing municipal authorities had to manage. Indeed it has been an issue for several previous Olympic Games, including Los Angeles and Athens.
Studies show that the special measures taken for the Olympic Games - in conjunction with weather conditions in August - led to reductions in carbon monoxide (CO) by 47 per cent; nitrogen dioxide (NO2) by 38 per cent; volatile organic compounds (VOC) by 30 per cent; particulate matter (PM10) by 20 per cent; and sulphur dioxide (SO2) by 14 per cent.
The Games also "greatly increased public awareness of air quality" in Beijing, leading the public to press for continued efforts to sustain the improved quality of life experienced during the Games.
However, the report notes that "there remains significant room to improve Beijing's air quality" - the PM10 standard in the city remains a challenge.
The Beijing organizers'comprehensive measures to cut emissions and improve public transportation in the sprawling city included a switch more stringent vehicle emission and fuel quality standards in 2008, in line with Euro III and Euro IV emission standards.
The report concludes that the Games played a crucial role in Beijing's transition to more stringent vehicle standards and smoothed the way for the Beijing authorities and China's Central Government to implement future sustainable transport initiatives.
However congestion remains a challenge, with 1,000 new vehicles registered daily. The authors recommend that the city further expand its public transport capacity by building more subways and railways, and consider policy options including tax cuts for clean vehicles and congestion fees in crowded areas.
Reducing Beijing's dependence on coal and improving energy efficiency and air quality were a priority for the Game organizers.
The city of Beijing continues to rely on coal for 40 per cent of its energy consumption, and the report recommends "a more deliberate effort to reduce coal consumption in Beijing, in tandem with increasing the supply of clean and renewable energy".
Protecting Beijing's limited water supply was a key objective of the organizers, and the report concludes that the measures they took "serve as an international example of creative and aggressive measures that can be taken to minimize water use, maximize the efficient use of existing water resources and protect the water ecosystems".
To build on these achievements, the report recommends that the Beijing Municipal Government ensure that innovative water-efficiency measures (including storm water capture and storage) be increasingly applied throughout the city, especially in all new buildings and developments.
Beijing achieved all of its commitments in the area of waste - waste classification and recycling goals were exceeded, and the in-venue recycling rate was 23 per cent higher than the committed level.
Green venues
The report praises Beijing's "remarkable results in incorporating an array of environmental elements into the planning, construction and management of Games venues".
Areas where there is room for improvement include sustainable forestry - with a lack of mandatory timber purchasing guidelines for the Beijing venues - as well as the lack of a systematic environmental footprint analysis to assess the benefits of the green measures.
The IOC should encourage future host cities to invest in new and innovative approaches in energy efficient design and water-saving technologies, and to adopt mandatory purchasing policies for Games-related construction materials.
Climate neutrality
For the first time for a global sports event, Beijing's climate neutrality programme took into account the carbon emissions from international flights. However, the lack of a clear methodology and reliable figures make it difficult for UNEP to assess the carbon footprint of the Games and the impact of the measures to mitigate and offset the emissions.
The report says organizers of upcoming major sports events should be encouraged to seriously look at their carbon footprint and analyze primary data from events. It would also be useful for the IOC and other sport organizations to come up with a harmonized standard for addressing the issue of carbon neutrality in sport.
Relations with Non-Governmental Organizations
The report says the Beijing Organizing Committee missed an opportunity to fully engage with NGOs and benefit from their expertise from the outset of the Games preparations. The organizing committee only began engaging NGOs from the end of 2006, by which time the construction of most Games facilities was nearing completion and many policies were already in place.
The authors say it would be useful for the Beijing Municipal Government to engage NGOs in discussions on how to carry forward and reinforce the full range of environmental measures undertaken for the Games.
Notes to Editors
This UNEP report follows a memorandum of understanding between the organization and the Beijing 2008 Organizing Committee of the Olympic Games (BOCOG) in 2005 in which UNEP offered assistance to achieve environmentally friendly Games.
As part of this agreement, UNEP conducted a review and published a report on the environmental performance of the Beijing Olympic Committee in October 2007. http://www.unep.org/sport_env/Activities/beijingConf07/media/index.asp
UNEP also pledged to conduct this final assessment to review the impact of the environmental measures that were implemented by Beijing for the Games. Beijing 2008 Olympic Games - Final Environmental Assessment is available on the UNEP website www.unep.org
This report, whose findings have been shared with the International Olympic Committee and the BOCOG, is based on several field visits to China by UNEP staff and experts from 2007 to December 2008, as well as the analysis of data from the Beijing Municipal Government and BOCOG, and liaison with relevant NGOs.
For more information on the UNEP Governing Council, please visit http://www.unep.org/gc/gc25/
For more information please contact:
Nick Nuttall, UNEP Spokesperson, Office of the Executive Director, on Tel: +254 20 762 3084; Mobile: 254 733 632 755 or when traveling +41 795 965 737; E-mail: firstname.lastname@example.com
Theodore Oben, UNEP Chief of Sports & the Environment, on Tel: +254 724 255 247 or E-mail: email@example.com
Anne-France White at Mobile: +254 (0)728 600 494
or E-mail: email@example.com
Shereen Zorba at Mobile: +254 (0)721 210 571
or E-mail: firstname.lastname@example.com
Further Resources
Independent Environmental Assessment: Beijing 2008 Olympic Games
Beijing 2008 Olympic Games - An Environmental Review
UNEP Governing Council
UNEP Sports and Environment Website
Follow Us
Keep up to date with UNEP events on Facebook, Twitter and You Tube.


text:
Little Things Mean A Lot at this time of year. With Valentine's Day right around the corner and all the items in the stores, you can do Romance on a Budget and here are 10.
Here are a few suggestions for both with love in the air.
1. Gift wrap a wishbone in a jewelry box. Send it to her with a note that says, "I wish you were here."
2. Unplug the TV. Put a note on the screen saying, "Turn me on instead."
3. Go through revolving doors together.
4. Keep candles in the car. Eat dinner by candle light the next time you go to McDonald's.
5. Run your hands under hot water before joining your partner in bed(!)
6. Look—no, gaze—into each other's eyes more often.
7. Write him a little love note. Insert it into the book he's reading.
8. Pick wildflowers from a field or the side of the road during the year.
9. The relationship secret that's not really a secret: Express your love in lots of little ways.
10. Make a GIANT greeting card out of a big cardboard box.
Better yet you can do many things for your special someone during this lovely time and join us at the Franklin Victorian Bed and Breakfast in Sparta, WI for a Special time with the one you love all year long. We look forward to making your time Special with Love.


text:
Bought 26
Save up to 43%
Don't just take our word for it, see what people are saying on Trip Advisor:
'"I've had the coziest sleep in Apsis Porta Marina and can't imagine my vacation being any better than what it was!"'
Reviewed: 27 March 2014
'Highly recommend!"'
Reviewed: 21 December 2013
'"It was a nice place to stay"'
Reviewed: 27 November 2013
A city packed to the rafters with ground-breaking art, architectural feats and gourmet cuisine - it's hardly surprising that Barcelona consistently lures tourists back time and time again…
That's why today we're offering you a 2-night 4* Barcelona break including flights from £139pp, saving you up to 43% off iFly's prices.
Just a short flight from London Gatwick or Stansted, Barcelona offers discerning travellers everything from balmy climes to mesmerising architecture and relaxed tapas bars. With today's deal you'll stay at the 4* Porta Marina Hotel, a friendly base renowned for its spacious rooms and light and airy décor. As well as staying in a a sizeable double or twin room, you'll also be kitted out with a TV and tea and coffee making facilities - perfect for kicking back, relaxing and drinking a cuppa as you plan your time in the vibrant city.
Those who have never experienced the bright lights and bustling holiday ambience of Barcelona should kick off their stay with a visit to perhaps the world's most famous unfinished cathedral - a Gaudi masterpiece that you just can't fail to ignore. Other highlights include the Picasso Museum, the imposing Museum of National Art and the ever-colourful Park Guell. Framed by sandy beaches and bursting with market stalls, boutiques and the smells of fresh seafood, the city is pure unadulterated holiday heaven!
The deal can also entitle you to a Wowcher Wallet Credit. Click here for more info.


text:
Canned crabmeat comes in different styles and varieties, including lump (most expensive), white and claw (least expensive). We chose the white for the best appearance in this elegant appetizer. Frozen crabmeat can be used instead of the canned, if desired.
Chill the tomato-cucumber mixture up to 1 hour before spooning into glasses. A bonus is the flavors will have blended more.


text:
Jessica Simpson Keenanx Oxford Pump
Sorry, this item is sold out!
Looks like you've got more shoe shopping to do - have fun!
Looks like you've got more shoe shopping to do - have fun!
Item # dsw12prod4230116
Give yourself an update with the fabulous Keenanx. Jessica Simpson brings us this exciting, trendy pump to pair with your favorite looks for ultra chic style.
- Faux leather & hair upper
- Upfront lacing
- ¾" platform, 4¼" covered heel
- Synthetic sole
- Imported
- View more Women's Jessica Simpson Shoes
By: Lovequist
Location: New Jersey
April 18, 2013
Beautiful Bootie
"The two-tone texture is striking. They look terrific with bootcut and skinny jeans. I find however that it does put a bit too much pressure on the front ball of my foot."
By: BeautyzPain
Location: Marietta, GA
March 24, 2013
"I bought these to replace cute black boots I had! I bought them in a slightly bigger size because I want to wear them with thick leggings. These are perfect! They're comfy and work great with any outfit!"
Click anywhere to close this.


text:
Retro sounding funk track with distorted bass line, funky drums, Hammond licks and some horns. The track loops seamlessly and It has both wav and mp3 version. "Fuzzy Logic Groove" is suitable for film scenes, trailers, intros for TV shows, commercials etc.
If you are a videohive or activeden author, feel free to use the preview file
Similar Items


text:
UCL Australia
- About us
- Study
- Research
- Current Students
- News
Contact Us
UCL Australia
Torrens Building
220 Victoria Square
South Australia
Tel: +61 8 8110 9960
For all admissions enquiries please click here.
For all other enquiries please use the form below:
Directions to UCL Australia
By Air
Take a taxi from the airport to Victoria Square. The Torrens Building is located on the North East section of the Square on the corner of Wakefield Street.
By Bus
Victoria Square is well served by a number of local bus routes, please visit: Adelaide Metro for further information.
By Tram
The service between Glenelg and North Terrace passes through Victoria Square, please visit: Adelaide Metro for further information.
UCL International Office
If prospective students have any questions relating to programmes offered by UCL in London, please contact:
e-Mail: email@example.com
Phone: +44 20 7679 7765