http://climatedata.blprnt.com/ Data Visualization & Climate Reality en-US Thu, 29 Jul 2010 22:58:00 +0000 http://bbpress.org/?v=1.0.2 q http://climatedata.blprnt.com/search.php http://climatedata.blprnt.com/topic/matlab#post-44 Sun, 13 Dec 2009 22:12:18 +0000 Arenamontanus 44@http://climatedata.blprnt.com/ <p>I wrote a little script to read the data into Matlab for further processing. Nothing particularly clever, mainly a bit of file management and string processing to extract data and convert it into arrays. Some basic visualisations can be done.<br /> Right now it just deals with each station separately, but the next step involves merging the data in various ways. </p> <p>Not sure if it works in Octave.</p> <p><code><br /> % Matlab code to load Land surface climate station records<br /> % Assumes the data is unzipped into a subdirectory named Data.<br /> % Iterates through the subfolders, loading the data station by station<br /> %<br /> % Anders Sandberg<br /> subdirs=dir('Data');<br /> files={};</p> <p>llong=[];<br /> llat=[];<br /> hheight=[];</p> <p>for i=1:length(subdirs)<br /> if (subdirs(i).name(1)~='.')<br /> datafiles=dir(strcat('Data/',char(subdirs(i).name)));<br /> for j=1:length(datafiles)<br /> if (datafiles(j).name(1)~='.')<br /> filename = strcat('Data/',char(subdirs(i).name),'/',char(datafiles(j).name));</p> <p> fid=fopen(filename,'r');<br /> %tline = fgetl(fid);<br /> %sscanf(tline,'%d'),</p> <p> while (1)<br /> tline = fgetl(fid);<br /> if (strcmp(tline,'Obs:')) break; end<br /> C=textscan(tline,'%s%s', 1,'Delimiter','=');<br /> if (strcmp(C{1},'Number')) id=C{2}; end<br /> if (strcmp(C{1},'Name')) name=C{2}; end<br /> if (strcmp(C{1},'Country')) country=C{2}; end<br /> if (strcmp(C{1},'Lat')) lat=C{2}; lat=str2num(lat{1}); end<br /> if (strcmp(C{1},'Long')) long=C{2}; long=str2num(long{1}); end<br /> if (strcmp(C{1},'Height')) height=C{2}; height=str2num(height{1}); end<br /> if (strcmp(C{1},'Start year')) startyear=C{2}; end<br /> if (strcmp(C{1},'End year')) stopyear=C{2}; end<br /> if (strcmp(C{1},'First Good year')) goodyear=C{2}; end<br /> if (strcmp(C{1},'Source ID')) sourceid=C{2}; end<br /> if (strcmp(C{1},'Source file')) sourcefile=C{2}; end<br /> if (strcmp(C{1},'Normals source')) normalsource=C{2}; end<br /> if (strcmp(C{1},'Normals source start year')) normalstart=C{2}; end<br /> if (strcmp(C{1},'Normals source end year')) normalend=C{2}; end<br /> if (strcmp(C{1},'Normals')) normals=C{2}; nromals=str2num(normals{1}); end<br /> if (strcmp(C{1},'Standard deviations source')) stdsource=C{2}; end<br /> if (strcmp(C{1},'Standard deviations source start year')) stdstart=C{2}; end<br /> if (strcmp(C{1},'Standard deviations source end year')) stdend=C{2}; end<br /> if (strcmp(C{1},'Standard deviations')) stds=C{2}; stds=str2num(stds{1}); end<br /> end</p> <p> data=textscan(fid,'%d%f%f%f%f%f%f%f%f%f%f%f%f');</p> <p> % Vector of years for this station<br /> year=double(data{1});<br /> % Matrix of temp data<br /> bdata=[data{2} data{3} data{4} data{5} data{6} data{7} data{8} data{9} data{10} data{11} data{12} data{13}];<br /> % Vector of temp data<br /> sdata=bdata'; sdata=sdata(:);<br /> % Vectors of months and years<br /> smonth=repmat(1:12,1,length(year));<br /> syear=[]; for k=1:length(year); syear=[syear ones(1,12)*year(k)]; end</p> <p> N=length(sdata);</p> <p> % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%VISUALISATIONS<br /> % Comment/uncomment as needed</p> <p> % Remove nonexistent data<br /> sdata(find(sdata&lt;-98))=NaN;</p> <p> % plot full data serially<br /> %plot(syear+(smonth-1)/12, sdata)</p> <p> % plot year cycle<br /> %plot(sdata(1:(N-3)),sdata(4:N))</p> <p>% % Plot colored year cycle<br /> % clf<br /> % hold on<br /> % for ii=1:(N-4)<br /> % co=ii*[1 1 1]/N;<br /> % plot([sdata(ii) sdata(ii+1)],[sdata(ii+3) sdata(ii+4)],'LineWidth',2,'Color',co);<br /> % end<br /> % set(gca,'Color',[153 136 85]/255)<br /> % xlabel('T(t)')<br /> % ylabel('T(t+3)')<br /> % V=axis;<br /> % title(strcat(char(name),' / ',char(country)),'FontSize',20)<br /> % </p> <p> % make map<br /> %if (height&lt;-100) height=0; end; % remove missing heights<br /> %plot3(long,lat,height,'.')<br /> %text(long,lat,height,strcat(name,' ',country));</p> <p> % plot monthly deviation from mean temperature<br /> %mm=mean(sdata(find(~isnan(sdata))));<br /> %plot(syear+(smonth-1)/12, sdata-mm);</p> <p> % Plot a polar diagram of temperatures<br /> clf<br /> hold on<br /> singleplot=0; % single color or multicolor<br /> mi=-50;<br /> ma=50;<br /> % Draw background lines<br /> theta=2*pi*(0:100)/100;<br /> for r=-30:10:30<br /> R=0.1+0.9*(r - mi)/(ma-mi);<br /> if (R&gt;0)<br /> plot(R.*cos(theta),R.*sin(theta),'k','Color',[0.7 0.7 0.7])<br /> text(R+.025,0+.025,num2str(r),'FontSize',8)<br /> end<br /> end<br /> monthname=['Jan';'Feb';'Mar';'Apr';'May';'Jun';'Jul';'Aug';'Sep';'Oct';'Nov';'Dec'];<br /> for mo=1:12<br /> tm=-2*pi*(mo-1)/12;<br /> plot([0 Rm*cos(tm)],[0 Rm*sin(tm)],'k','Color',[0.7 0.7 0.7]);<br /> text(1.05*Rm*cos(tm),1.05*Rm*sin(tm),monthname(mo,:),'HorizontalAlignment','center');<br /> end<br /> theta=-(2*pi*(smonth-1)/12)';<br /> R=0.1+0.9*(sdata - mi)/(ma-mi);<br /> Rm=1;<br /> if (singleplot==1)<br /> plot(R.*cos(theta),R.*sin(theta))<br /> else<br /> for ii=1:(N-1)<br /> co=ii*[1 1 1]/N;<br /> plot([R(ii)*cos(theta(ii)) R(ii+1)*cos(theta(ii+1))],[R(ii)*sin(theta(ii)) R(ii+1)*sin(theta(ii+1))],'Color',co,'LineWidth',2);<br /> end<br /> end<br /> axis([-1.1 1.1 -1.1 1.1])<br /> axis square<br /> axis off<br /> title(strcat(char(name),' / ',char(country)),'FontSize',20)</p> <p> % Make unreliable mean year plot<br /> % bdata(find(bdata&lt;-98))=NaN;<br /> % plot(year, mean(bdata'))</p> <p> drawnow<br /> pause</p> <p> %%%%%%%%%%%%%%%%%%%%%%%%%<br /> % Collect data across stations<br /> llong=[llong long];<br /> llat=[llat lat];<br /> hheight=[hheight height];</p> <p> fclose(fid);<br /> end<br /> end<br /> end<br /> end<br /> </code> </p> http://climatedata.blprnt.com/topic/seasonal-variations-per-areas#post-42 Sun, 13 Dec 2009 13:36:30 +0000 mgilbir 42@http://climatedata.blprnt.com/ <p>I just gave a shot at the dataset and came up with my first Processing visualisation.</p> <p><object width="500" height="375"></p> <param name="allowfullscreen" value="true" /> <param name="allowscriptaccess" value="always" /> <param name="movie" value="http://vimeo.com/moogaloop.swf?clip_id=8150082&amp;server=vimeo.com&amp;show_title=0&amp;show_byline=0&amp;show_portrait=0&amp;color=00adef&amp;fullscreen=1" />&lt;embed src="http://vimeo.com/moogaloop.swf?clip_id=8150082&amp;server=vimeo.com&amp;show_title=0&amp;show_byline=0&amp;show_portrait=0&amp;color=00adef&amp;fullscreen=1" type="application/x-shockwave-flash" allowfullscreen="true" allowscriptaccess="always" width="500" height="375"&gt;&lt;/embed&gt;</object></p> <p><a href="http://vimeo.com/8150082">Global average temperature evolution 1701 - 2009</a></p> <p>It aims to represents how the temperature has evolved over time in different regions of the Earth:</p> <ul> <li>North represents the data recorded by stations between the Arctic Circle and the Tropic of Cancer. </li> <li>Tropic represents the data recorded by stations between the Tropic of Cancer and the Tropic of Capricorn. </li> <li>South represents the data recorded by stations between the Tropic of Capricorn and the Antarctic Circle. </li> </ul> <p>I am preparing a blog post with some further analysis of the data. All feedback is very much appreciated :) </p> http://climatedata.blprnt.com/topic/geospatial-distribution-of-weather-stations#post-46 Mon, 14 Dec 2009 02:02:53 +0000 gka 46@http://climatedata.blprnt.com/ <p>I made a little animation showing the geospatial distribution of weather stations over time. The temperature range is from -70°C (dark blue) over 0° (white), 10°C (yellow), 32°C (red) to 40°C (dark red). The video goes through 3 loops showing the mean temperature (MT), the summer mean temperature (SMT) and the winter mean temperature (WMT).</p> <p>See the video: <a href="http://vimeo.com/8160362" rel="nofollow">http://vimeo.com/8160362</a></p> <p>The second charts shows the percentage of weather stations in certain latitude regions over time:</p> <p><img src="http://vis4.net/blog/wp-content/uploads/2009/12/weather_chart.png" alt="" /></p> <p>Data source: <a href="http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html" rel="nofollow">http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html</a> </p> http://climatedata.blprnt.com/topic/climate-change-visualisation#post-25 Fri, 11 Dec 2009 06:53:17 +0000 Flinklabs 25@http://climatedata.blprnt.com/ <p>We've just put together a 3D globe style visualisation of the data. More details including source code, an image and a video are available at <a href="http://flinklabs.com/projects/climatedata/" rel="nofollow">http://flinklabs.com/projects/climatedata/</a> </p> http://climatedata.blprnt.com/topic/links-to-datavis-tools#post-31 Fri, 11 Dec 2009 18:28:14 +0000 eggsyntax 31@http://climatedata.blprnt.com/ <p>For anyone getting started in datavis, the plethora of tools can be intimidating. Here are three collections I've found very useful in choosing:</p> <p><a href="http://flowingdata.com/2008/10/20/40-essential-tools-and-resources-to-visualize-data/" rel="nofollow">http://flowingdata.com/2008/10/20/40-essential-tools-and-resources-to-visualize-data/</a><br /> <a href="http://www.wikiviz.org/wiki/Tools" rel="nofollow">http://www.wikiviz.org/wiki/Tools</a><br /> <a href="http://www.readwriteweb.com/archives/the_best_tools_for_visualization.php" rel="nofollow">http://www.readwriteweb.com/archives/the_best_tools_for_visualization.php</a></p> <p>Good luck! </p> http://climatedata.blprnt.com/topic/here-is-the-data#post-2 Wed, 09 Dec 2009 06:10:55 +0000 blprnt 2@http://climatedata.blprnt.com/ <p>Here is the data released today by the Met Office:</p> <p><a href="http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html" rel="nofollow">http://www.metoffice.gov.uk/climatechange/science/monitoring/subsets.html</a> </p> http://climatedata.blprnt.com/topic/image-posting-now-enabled#post-36 Sat, 12 Dec 2009 02:09:37 +0000 blprnt 36@http://climatedata.blprnt.com/ <p>We have joined the 21st century - image posting is now enabled in the forum. Feel free to edit your posts as needed. I'll see if I can get vimeo embed allowed, as well... *EDIT - Vimeo embeds are now enabled, too!</p> <p><a href="http://www.flickr.com/photos/blprnt/4176309538/" title="Met Office Climate Data - Month by Month (January) by blprnt_van, on Flickr"><img src="http://farm5.static.flickr.com/4037/4176309538_96cb0f59f3.jpg" alt="Met Office Climate Data - Month by Month (January)" /></a> </p> http://climatedata.blprnt.com/topic/climatedata-flickr-pool#post-33 Fri, 11 Dec 2009 21:00:44 +0000 blprnt 33@http://climatedata.blprnt.com/ <p>I've created a Flickr Pool:</p> <p><a href="http://www.flickr.com/groups/1270263@N24/" rel="nofollow">http://www.flickr.com/groups/1270263@N24/</a></p> <p>Please add any images and videos to this pool! I'll be making a widget for the main page to show the recent images from this pool. </p> http://climatedata.blprnt.com/topic/interactive-visualization-from-eagereyesorg#post-32 Fri, 11 Dec 2009 20:54:30 +0000 blprnt 32@http://climatedata.blprnt.com/ <p>Robert Kosara has posted a nice interactive visualization of the Met Office data:</p> <p><a href="http://eagereyes.org/data/interactively-explore-climate-data#comment-2065" rel="nofollow">http://eagereyes.org/data/interactively-explore-climate-data#comment-2065</a> </p> http://climatedata.blprnt.com/topic/flexflare#post-30 Fri, 11 Dec 2009 18:25:15 +0000 eggsyntax 30@http://climatedata.blprnt.com/ <p>I've been enjoying doing climate data visualizations using the combination of Adobe's Flex development environment and the Flare visualization API. Flare has equivalents in other languages as well (Prefuse for Java, Protovis for Javascript), but I find Flex to be a pleasure to work with. </p> http://climatedata.blprnt.com/topic/another-dataset#post-29 Fri, 11 Dec 2009 18:22:35 +0000 eggsyntax 29@http://climatedata.blprnt.com/ <p>Hi,</p> <p>I work with the Climate Reference Network ( <a href="http://www.ncdc.noaa.gov/crn/" rel="nofollow">http://www.ncdc.noaa.gov/crn/</a> ), which provides data similar to that released by the Met Office, but on a national scale for the U.S. We attempt to provide extremely high-quality data (using triplicate measurement for temperature and precipitation, for example, and extensive quality control). Data can be explored at <a href="http://www.ncdc.noaa.gov/crn/observations.htm" rel="nofollow">http://www.ncdc.noaa.gov/crn/observations.htm</a> , and full datasets can be downloaded at <a href="ftp://ftp.ncdc.noaa.gov/pub/data/uscrn/products/" rel="nofollow">ftp://ftp.ncdc.noaa.gov/pub/data/uscrn/products/</a> . Just wanted to let y'all know that it's out there as another resource, and I'd be curious to hear about it if anyone does any visualization work with our data.</p> <p>-Egg Davis </p> http://climatedata.blprnt.com/topic/get-to-know-the-data#post-19 Thu, 10 Dec 2009 19:35:38 +0000 rockitbaby 19@http://climatedata.blprnt.com/ <p>Thats what I did today. I wrote this visualization to get to know the data, not to show anything special.</p> <p><a href="http://vimeo.com/8102161" rel="nofollow">http://vimeo.com/8102161</a></p> <p>Source:<br /> <a href="http://rockitbaby.de/static/climatedata_01.zip" rel="nofollow">http://rockitbaby.de/static/climatedata_01.zip</a></p> <p>Based on the MySQL-dump, thanks a lot.</p> <p>Feel free to play around and send me feedback.<br /> Michael </p> http://climatedata.blprnt.com/topic/data-exploration-monthly-sets#post-26 Fri, 11 Dec 2009 08:21:49 +0000 blprnt 26@http://climatedata.blprnt.com/ <p><a href="http://www.flickr.com/photos/blprnt/4176309538/" title="Met Office Climate Data - Month by Month (January) by blprnt_van, on Flickr"><img src="http://farm5.static.flickr.com/4037/4176309538_96cb0f59f3.jpg" alt="Met Office Climate Data - Month by Month (January)" /></a></p> <p>I spent a few hours with the data this evening. Mainly the purpose of these images is to get an idea of the scope of the data. I thought it would be helpful to break the records down into monthly sets. Also I sorted the stations by latitude, which groups like measurements together.</p> <p>I want to do some more exports like this focusing on certain geographical areas (N. Africa, for instance) to see if I can identify some trends...</p> <p><a href="http://www.flickr.com/photos/blprnt/sets/72157622978635070/" rel="nofollow">http://www.flickr.com/photos/blprnt/sets/72157622978635070/</a> </p> http://climatedata.blprnt.com/topic/processing-libraries-feature-requests#post-5 Wed, 09 Dec 2009 18:59:04 +0000 blprnt 5@http://climatedata.blprnt.com/ <p>Hi All,</p> <p>I'm going to work at putting together some simple Processing (<a href="http://processing.org" rel="nofollow">http://processing.org</a>) libraries. These libraries will read the Met Office data from some source (ideally a central online source that would allow the data to be updated with new sources as needed).</p> <p>I'm placing this thread here as a stub and will update it with details as I get a chance to work with the data a bit and get a fee for what the Processing libraries should be able to do.</p> <p>-Jer </p>