IDEA: Need short audio / video recordings triggered from Browser event

[Oilman]

Greetings-

This is my first post / request on this site even though I have used (some on a regular basis) alot of programs you all have created. I think this site / you as a group are an amazing resource and wealth of talent. I also want to give Kudos to Mouser for the individual attention and interaction he has given to me over the last couple weeks.

Now that I have schmoozed and and buttered up to y'all a little.... Here is my situation, and If anything about this request doesnt fit into guidelines please let me know cause as mentioned this is the first (hopefully of a couple more in the future)

I am trying to take a 15 second screen recording (Audio for full 15 seconds along with either a screen shot or 15 seconds of video)  from my office phone system every time a call is routed to a certain browser / extension. Basically I want to be able to monitor it for a couple days then review the data. Ive played around with several ideas and I think the simplest would be to trigger a screen audio / video  recorder for a couple seconds to record the clients profile record (which pops up via a web browser) along with a couple seconds of audio that comes from same browser then pause it waiting for the next record / call to arrive.

Then I can review the recording showing only activity and not 2 hours (or days) of blank screen between activity.

After I mutilated some AHK scripts I came to the conclusion (12th conclusion on this little project) that AHK isnt the way to tackle it. Since I am interacting with a webpage it seems it could be much more elegant to accomplish this by working directly with the HTTP request / responses Unfortunately that is another skill that I know just enough about to envision an end product but zero skill to accomplish it.

In my limited research it seems like it wouldnt be overly difficult since there are not many variables, actually just 3 steps.

1: when incoming HTTP shows "X record is online" in the string press the F9 key (starts camtasia recorder)
2: wait 15 seconds then press the F10 key (pauses camtasia recorder)
3: send HTTP response to close record "X"
4:Wait for next incoming HTTP saying "X record is online"

I have the actual HTTP strings at the office but basically "X" is the clients database record # with a standard phrase that is something like "live agent transfer" then the disconnect string is basically the same but to close the record.

What I cant find / dont know if feasible is to trigger the F9 and F10 keys from incoming headers. Does anyone know of any programs that can bridge between browser requests and key function? Or any better solution to accomplishing this sequence?

Better yet if any of you are able to make something like this happen I dont mind paying someone what its worth to get it done right vs me beating my head against a wall for another couple weeks.

Even better still if someone wants to be creative and include some way to VR and transcribe the event along with the recording I would be willing to cover that as well. (but the VR and transcribe isnt necessary)

I also have a couple similar projects that I need done but this one is the starting gate for the others.

Thank you for taking the time to read and consider this. I am completely open to ideas / suggestions / etc.

If anyone needs additional details I made a screen recording of the interaction including one with a HTTP recorder open so you can see the sequence in more detail.

Not sure if i can put my email here so if interested shoot me a PM with what you would expect / need for your time to complete and anything additional from my end to make this program a reality and I have no problem taking care of it as early as Monday.

Cheers boys and girls have a safe Sat night.

Matt-

[Ath]

Hi Matt,
Welcome to the donationcoder forum.
Not many guidelines here, but a friendly approach and clearly stated questions is going to get you almost anywhere you want to go
(yup, you got that covered )

A few questions:
- What application is triggering/shooting the urls into your browser?
- What webbrowser(s) are you using?

I think I can enhance my WinButtons project with the features you require here, but detecting an url in some browsers is kinda 'black magic' it seems, and there are no timers in WB yet, but those are the kind of challenges I can use atm.

[Renegade]

I won't be able to help much as we're traveling right now, but here's my quick take on it.

From reading your request, this is the kind of approach that I think would probably work best (I'd use C# as it provides low-level tools, and I don't know if AHK could do that):

1. Sniff all network traffic. There are quite a few articles on doing that. Here are some:

http://www.codeproject.com/Articles/17031/A-Network-Sniffer-in-C
http://stackoverflow.com/questions/2953575/how-to-sniff-local-outgoing-network-traffic-in-net-without-using-pcap

This is really the hardest part of what you're asking. Everything else is pretty simple. Doing this in C# or some similar language provides the benefit of having a HUGE amount of control. If forced to use a browser plug-in, then things change a bit and you lose a lot of power/control, but that may not be necessary. (I'm not sure how/if a plug-in can run another piece of software. AV software does it, but I'm not familiar with "under the hood" there, and don't know the APIs or amount of effort required -- sometimes these things are trivial, and other times they are made deliberately difficult to prevent people from easily writing malware, i.e. they make malware writing only available to professional malware authors. )

Oh, 1 other thing about this... It could trigger AV software as this is the kind of behaviour that malware typically depends upon. (Similar for "SendKeys" below.)

2. When a TCP/IP HTTP connection comes across, read the headers (presumably to look for a specific URL or domain that you're looking for) or read the entire document if text in the body triggers what you want, and then trigger the recorder. The problem here is that this won't work if you're using HTTPS. In that case, nix #1 above and write a custom browser plug-in to monitor what you want and then trigger the recorder from there.

The conditions here would be the most important thing. Above you've not really given enough information to comment on how to do this. e.g. Are you looking for a URL? Domain name? Text inside of a URL, such as "watch=videoGUID"? Some text inside of the web page itself? Multiple identifying items? Perhaps the event is an object inside of the browser, like a Flash program that can be reliably detected?

You mention "call" and a few other things in there. I'd ask if what you're trying to record is coming across RTSP, which could help in detection or recording in some cases.

3. Triggering the Recorder: Lots of ways to do this. Either a built-in recorder, or like you mention, Camtasia. This is as simple as sending the F9 or F10 key events. i.e. SendKeys.Send("{F9}").

Now, the problem with using Camtasia is that you'd already have to have it running, or the program would need to detect that Camtasia is running, and then trigger it. However, if it isn't running, by the time the program could run Camtasia, your event would be over. So, you'd need a built-in recorder to deal with that case. That's not too bad though as there are a few code bases out there that are pretty much ready to go to do that.

The silly thing then is that it would make Camtasia redundant. However, that's not a bad thing. With Camtasia, would you really want to have it constantly recording and stopping? What happens if something goes wrong and it hasn't saved the events? Ouch... By recording in the program itself, it's easier to simply save the recordings with a timestamp so that you can review all of them later. That way, if there's a power outage or something, at least you have previous recordings already saved.

The other option is to do some kind of software automation with Camtasia, which can be messy. You're likely to run into problems at some point there.

etc.

If you only wanted to record audio (I'm not really a video guy, but I know that there are relatively simple ways to record a screen), BASS would be a good choice to manage that. The Windows audio stuff is ok for some things, but BASS simply rocks.

Anyways, those are just a few of my initial thoughts on the topic. I'm not sure if they help you or anyone else who could manage to get that done for you.

[Oilman]

Good Monday-

Im going to respond quick right now cause i have a flight to catch then i will re-read and see if I missed any answers to your questions. But first I would like to thank you for the quick initial replies.

1: It does not matter to me what browser (explorer chrome firefox). I have all of them and use each for situations that levy on their individual strengths.

2: I did expect that I would already have to have camtasia or similar program running. This would be set up on a dedicated computer so it wont have the clutter of any other inputs.

3: Even though this is a website that is being logged into it is a subscribed site with a pin and password.

Lets see if I can be more specific without making it worse. (And I do have a 10 min screen recording of running the process manually with  HTTP live open so that it shows the incoming / outgoing requests) but here is an overview:

We have a financial adviser firm that works strictly with people who retire from AT&T Exxon Verizon and Merck. We have independent advisers across the country that work under our name. As part of an effort to make sure all advisers are being client orientated we have a followup system where we call our clients every couple months and ask them how they are enjoying retirement etc, while we get down to the question of has "joe adviser" been able to help you with everything and overall give you warm fuzzy feelings or did he invest you into a bunch of products that you dont understand and then stop answering your calls. (better never happen)

So one tool we use to contact these clients is called MOJO. It is a CRM / Dialer hybrid that isnt cheap but its a very efficient way to take the load of new clients and as they are called it connects them directly to a quality control rep who rates the call and pushes one of the buttons which sorts that client record into one of 4 buckets from good / happy with "joe adviser" he has everything under control no need to elevate, all the way down to: transfer call to a supervisor right now cause they are thinking of closing accounts. (again better never happen)

So to run this we have QC personnel log into mojosells.com and after entering credentials that webpage opens up to pretty standard CRM welcome page there it stays static except on the HTTP you can see the client record number changing as it calls the client. Once a client answers is when the screen changes to that clients record and the HTTP send a header saying something like "record 00283746 Live answer" then after the call when the rep pushes the button to sort the record there is a HTTP header sent saying "record 00283746 closed by agent contacted" which tells MOJO to dial the next record.

Not much of that needs to be messed with because it is handled by live agents. But what if the client says "Everything is good but I talked to "joe advisers" secretary 2x last week about a new beneficiary form and no one has called me back"...

So we then call "joe adviser" and he says its in the mail to the client and he left the client a VM but didnt hear back, but will call after the weekend to make sure he got the paperwork".. Well this isnt a housefire emergency and it sounds like everything is under control but since it is our financial firms name this adviser is working under its good for compliance reasons to have the confirmation from the client that all is good in the world. So lets say next week we try to follow up a couple times and maybe leave a voicemail but dont hear back from the client, they finally filter down into a survey bucket that isnt high priority cases but still cant quite be closed out.

(Deep breath) So this same MOJO system can also run the auto attendant survey that says "was your case handled to your satisfaction press 1" etc

Well undoubtedly there is always mr client who thinks that "press one" means enter your phone number and try pressing the # key 6X then yell at your dog to stop barking at the mailman. Instead of hanging up just set the phone down while you go upstairs and ask your wife if your satisfied or not, and lastly see the dog you were yelling at, forget what you were going to ask your wife and leave the phone off the hook for a week while wondering why no one likes you enough to ever call.

All jokes aside, this is where the need for some automation comes in. if MOJO forwards a clients record to the extension to handle "unknown errors" I need to be able see what record it was and hear if anything was going on. usually the error was that during the persons voicemail greeting there is a keytone (happens alot) and so MOJO gets confused about what to do or when some one presses 1# to say they were satisfied they actually press 11# etc etc. BUT MOJO WILL NEVER HANG UP....So on errors it will keep line tied up. So I want a quick audio / video sample and automatic termination of the line.

I think it can be boiled down to some simple parts if everything is on a dedicated computer that is always logged into the system and has camtasia already running.

1: When HTTP header string shows "live answer" in the string, press F10 to take camtasia off of pause
2: Wait for ____ seconds while recording
3: Press F9 to pause camtasia
4: Send header that says "unlock contact"
5: wait for next "live answer"

It does not need to be video just anyway to see what record it was. but I like the camtasia integration for 2 reasons:

1: it combines both the screen and audio recording with the press of 1 key (again if already running which it would be)
2: by pausing between events and having all in one long file I can review in the camtasia studio and see in the wave where any actual audio is and as I forward to that section the video follows in sync.

I hope I helped and didnt make things more confusing. Sorry for any format / grammar errors I have started and stopped this several times while moving through lines.

I am more then happy to compensate for anyone's time, and to the group as a whole for providing an outlet where someone can request and receive input and great end product from developers. Its alot more difficult then I expected to find / deal with the ones who can actually write whats needed. I wont post the quotes I got from software sales reps who all wanted to write new programs from ground up. Ouch!




Follow up....I think these should be the 2 HTTP interaction to Start camtasia and then one to tell MOJO to close and move on, However This was not a transfered call I had to initiate it in order to be able to know when it was going to happen. But from what I see / remember the only diff is the other says agent transfer in the strings but the short urls are the same.





https://app101.mojosells.com/ajax/dialer/live_answer/

curl "https://app101.mojosells.com/ajax/dialer/live_answer/" -H "Host: app101.mojosells.com" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:26.0) Gecko/20100101 Firefox/26.0" -H "Accept: */*" -H "Accept-Language: en-US,en;q=0.5" --compressed -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "X-Requested-With: XMLHttpRequest" -H "Referer: https://app101.mojosells.com/main/?frame=/dial_list/1/" -H "Cookie: __utma=167056930.347717322.1433797123.1433797123.1433797123.1; __utmz=167056930.1433797123.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not"%"20provided); sessionid=04e7 58a42593cf4a2eb1bf038011ef84" -H "Connection: keep-alive" -H "Pragma: no-cache" -H "Cache-Control: no-cache" --data "contact_id=123678&line_id=29"



https://app101.mojosells.com/ajax/unlock_contact/

curl "https://app101.mojosells.com/ajax/unlock_contact/" -H "Host: app101.mojosells.com" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:26.0) Gecko/20100101 Firefox/26.0" -H "Accept: application/json, text/javascript, */*; q=0.01" -H "Accept-Language: en-US,en;q=0.5" --compressed -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "X-Requested-With: XMLHttpRequest" -H "Referer: https://app101.mojosells.com/main/?frame=/dial_list/1/" -H "Cookie: __utma=167056930.347717322.1433797123.1433797123.1433797123.1; __utmz=167056930.1433797123.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not"%"20provided); sessionid=04e758a42593cf4a2eb1bf038011ef84" -H "Connection: keep-alive" -H "Pragma: no-cache" -H "Cache-Control: no-cache" --data "contact_id=123678&line_id=-1&call_result=2&phone_id=134579&is_unlock=1&r=0.21513073083202972&duration=42&call_id=16540&profile_id=1&dial_session_history=607"


So Much For Making this a Quick Response.
Matt-