Skip the navigation
Opinion

STFW, fix Pipes, and repair Mac disks; voila!

By Mark Gibbs
November 4, 2009 08:46 PM ET

Network World - The next time someone asks you a question and you know they haven't bothered to do the obvious, to wit, look it up on Google, send them a link to Let Me Google That For You. For example, if someone were to write to me and ask if there's an archive of Gearhead columns, I might reply with "Try this link: http://tinyurl.com/yfg33ty". Voila!

Anyway, last week I was discussing my trials and tribulations attempting to get a list of numbers randomized (exactly why I wanted to do this is too complex to go into).

My solution was to use Yahoo Pipes to grab the random sequence from the service random.org using an HTTP proxy on my own server.

The reason for this labyrinthine methodology was that Pipes, being the good citizen it is, will not access remote content blocked by a robots.txt access control file -- exactly what random.org has. My proxy hasn't got a robots.txt file so when Pipes looks for the robots.txt file it won't see one: Problem solved.

Now here's an interesting thing: When I finally got my pipe working it tested fine. In the development interface I could monitor the RSS feed output and whenever I refreshed the monitor I could see a new sequence being returned. Foolishly I concluded that it was working correctly.

It turns out that Pipes has a caching algorithm that is only used when you request the pipe be run from the Internet; requests made from within the Yahoo Pipes site are always honored. It took me a goodly time to figure this out because, of course, I assumed that the problem lay with everything other than Yahoo Pipes.

Thus, should you externally request the output of a pipe more than once during an unspecified and undocumented time period set by Yahoo, only the result of the first request is returned. There have been quite a few support questions about this raised in both the Pipes forum and other places, but apparently Yahoo doesn't care to explain what's going on.

For all of you (countless thousands I suspect) struggling with this issue, there's a simple solution: When you define the arguments for the pipe's URL, add another argument that the pipe will do nothing with.

For example, let’s name this additional argument "fudge". When you call the pipe, just make the value of this additional argument something such as the time; thus: "fudge=10:33:45".

The format of the value doesn't matter as long you change it for each request. Now, when Yahoo Pipes gets a request for your pipe it will always assume that it is a new request and therefore run the pipe rather than pull the previous results from cache. Voila encore!

Originally published on www.networkworld.com. Click here to read the original story.
Reprinted with permission from NetworkWorld.com. Story copyright 2010 Network World, Inc. All rights reserved.
Additional Resources
Forrester Consulting - Optimizing Users and Applications in a Mobile World
WHITE PAPER
Solving application issues over the WAN requires careful consideration. Based on their independent research, Forrester Consulting offers recommendations on how to tackle application performance issues, insufficient bandwidth and the inability to quickly restore users in a disaster.

Read now.

Security KnowledgeVault
WHITE PAPER
Security is not an option. This KnowledgeVault Series offers professional advice how to be proactive in the fight against cybercrimes and multi-layered security threats; how to adopt a holistic approach to protecting and managing data; and how to hire a qualified security assessor. Make security your Number 1 priority.

Read now.

Cut Communications Costs Once and for All
WHITE PAPER
New IP-based communications systems are being deployed by small and midsized businesses at a rapid rate. Learn how these organizations are enabling faster responsiveness, creating better customer experiences, speeding office or mobile interactions, and dramatically reducing existing communications costs.

Read now.

Applications White Papers
Forrester Total Economic Impact (TEI) Case Study - Oracle
In this paper, Forrester Consulting examines the total economic impact and potential return on investment (ROI) realized by three Enterprise organizations as they...
The Hidden Truth About Virtualizing Business-Critical Applications
This IDG whitepaper highlights key findings based on the Quickpoll Survey conducted with more than 300 Enterprise and Commercial IT decision makers worldwide...
Top 10 Myths About Virtualizing Business-Critical Applications
Even though virtualization has brought positive change to enterprise IT over the last decade, some skepticism remains about how valuable virtualization can be...
Enterprise Java Applications on VMware: Unix to Linux Migration Guide
This guide focuses on key considerations for IT Architects who are in the process of migrating Java applications from UNIX to Linux as...
Virtualizing Tier 1 Applications: A Critical Step on the Journey Toward the Private Cloud  
This IDC white paper explains how much of the Enterprise IT community is at a crossroads in extending their journey to the private...
All Applications White Papers
Applications Webcasts
Live Webcast
Banish Poor Application Performance: Eliminate Business Disruptions, Increase End User Productivity
End User Experience, 30-Min Webinar
Wed. Feb. 22nd ~ 11 AM ET

Are you ready to gain the proactive ability to rapidly respond...
Apps QuickStart Series Part 2: Designing and Deploying SQL Server on VMware vSphere
Download this webcast to learn about the design considerations for virtualizing SQL workloads, performance and scalability information and high-availability options, as well as...
Apps QuickStart Series Part 1: Designing and Deploying Exchange 2010 on VMware vSphere
Download this webcast to learn the virtual hardware design considerations for Exchange 2010, deployment using the building block approach, options for high-availability and...
Virtualize Business-Critical Applications with Confidence
Virtualizing business-critical applications has become a key focus for organizations as they move along their virtualization journey. With the launch of VMware vSphere®...
Discover the Benefits of Virtualization for Federal Applications
Want to say goodbye to missed SLAs? VMware can help you virtualize mission-critical applications such as Oracle, MS Exchange and SharePoint to achieve...
Reduce Application Lifecycle Management Costs with VMware ThinApp
Traditional desktop application deployment and management is a time-consuming and costly endeavor for IT. From development to deployment, including help desk support, the...
All Applications Webcasts
Newsletter Sign-Up

Receive the latest news test, reviews and trends on your favorite technology topics

Choose a newsletter
  1. View all newsletters | Privacy Policy
IT Jobs