How to automatically open a page, click 'accept' button on page, then write content of next page to console

Question

0.00/5 (No votes)

See more:

I'm trying to write code in C# to go to the following page,
https://www.frbservices.org/EPaymentsDirectory/agreement.html[^]
the, click on the 'Agree' button to go to the next page. I then need to save the next page to a text file. (FedACHdir.txt)

So far, I have the following code, which doesn't quite work.
Any help would be appreciated.
Thanks.

C#

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Http;
using System.IO;
using System.Windows.Forms;
using HtmlAgilityPack;

namespace FedACH
{
    class Program
    {
        [STAThread]
        public static void Main(string[] args)
        {
            var request = (HttpWebRequest)WebRequest.Create("https://www.frbservices.org/
EPaymentsDirectory/agreement.html");
            request.Method = WebRequestMethods.Http.Post;
            request.ContentType = "application/x-www-form-urlencoded";
            using (var stream = request.GetRequestStream())
            {
                var buffer = Encoding.UTF8.GetBytes("agreementValue=Agree");
                stream.Write(buffer, 0, buffer.Length);
            }
            var response = (HttpWebResponse)request.GetResponse();
            string result = String.Empty;
            using (var reader = new StreamReader(response.GetResponseStream()))
            {
                result = reader.ReadToEnd();
            }
            Console.WriteLine(result);
        }
    }
}

Posted 10-Jun-15 15:39pm

Member 10373082

Updated 10-Jun-15 18:51pm

v4

Add a Solution

3 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

David A. Gray · Answer 1 · 2015-06-10T17:59:00

My suggestions are based on the assumption that the application in question is a console program, and that the data to be saved is in var stream, although most of my suggestions would be equally applicable to the code behind an ASP.NET form.

Among your first decisions is where you want to store the files, and how to name them to prevent collisions. The naming convention may depend partially upon the number of users who will be executing the application and dumping files into the repository of responses, and whether there is some easily accessible unique identifier from which a name can be constructed.

If multiple users are writing to the same repository, the file name must take into account its source (which machine saved it). If all machines belong to a single domain, the machine name can be read into a string from the Environment.MachineName property. Using the machine name also gives you a means of quickly tracing a file to its source if there is a problem with it.

With respect to constructing the rest of the name, there are several options.

1) If you start with the machine name, you can append a time stamp, such as System.DateTime.UtcNow.Ticks to make it locally unique. To avoid the risk of name collisions during the transition from Daylight Saving Time to Standard time, when the hour between 1:00 AM and 2:00 AM happens twice, back to back you must use UTC time. If you use time stamps, your computer's clock must keep very accurate time. This is best assured by having its clock checked frequently against a time server that is, itself, frequently checked against a Stratum 2 NTP (Network Time Protocol) server. Otherwise, excessive clock drift could lead to duplicate file names being assigned. Should this happen, the easiest way to resolve the collision is with a call to System.Threading.Sleep for a small amount of time; 100 milliseconds or so should be enough.

2) Another option for the rest of the file name is a sequence number. However, I don't especially like this approach because the sequence number must be securely stored somewhere, and the stored value must be reliably updated every time you create a new file, preferably before you create it. Updating the sequence number first eliminates the risk of reusing a number, should the machine crash before the stream is written to disk.

3) A third naming option is a GUID, preferably in Registry format. The advantage of this approach is that it doesn't matter where the machines are, or whether some machines have duplicate names. The down side is that there is no way to associate the GUID with the machine that generated it. In fact, the newest versions of the GUID generators are specifically designed to defeat such identification.

Finally, with the decisions behind you, writing boils down to a single method call, to System.IO.File.WriteAllBytes(pathString, buffer). When everything to be written is available at the same time, as it is in your case, this one method call gets the file opened, written, and closed cleanly.

I trust that you are aware that the using block and the WriteAllBytes method call should be protected by a try block.

David A. Gray · Answer 2 · 2015-06-15T10:24:00

Solution 2

Your example shows a console program. If the problem is actually in the code behind an ASPX form, then, yes, it will raise a security exception, because it is a grave security violation to use the console in a Web application.

Posted 15-Jun-15 10:24am

David A. Gray

David A. Gray · Answer 3 · 2015-06-15T18:55:00

I had another thought about your response indicating that you get a security exception. If the console program is just an example that you wrapped around your ASP.NET code to demonstrate your issue, then the security violation is likely caused by the fact that the user account under which ASP.NET pools run usually doesn't have permission to write or create files in any directory. That being the case, you will need to adjust the permissions on the directory where you intend to create the files, to enable that account to write in it.

I can't tell you the name of the account because it varies depending on the version of IIS and ASP.NET, and on how both are configured. The easiest way to adjust those permissions is through the IIS management console.

How to automatically open a page, click 'accept' button on page, then write content of next page to console

3 solutions

Solution 1

Solution 2

Solution 3

Add your solution here

Preview 0