[Solved] How to correctly read response from a WSProxy call to retrieve DE rows (more than 2500 using HasMoreRows) and update All Subs List

Andrew Asks: How to correctly read response from a WSProxy call to retrieve DE rows (more than 2500 using HasMoreRows) and update All Subs List
I have a Data Extension holding 80,000 rows of USA customer data. Within the DE is a field named “Country_Code” which sets a default value of “US” for everyone. My All subs list also has a custom attribute field named “Country Code”.

I’m trying to run a script on the parent BU that matches all contacts from the USA DE, on the All subs list, and if found update the Country Code to “US”.

I’m using the list update ssjs function:

Code:
var subscriber = {

"Attributes": {"Country_Code":Country_Code}};

var subObj = Subscriber.Init(sub_key);

var response = subObj.Update(subscriber);

Combined with a “Has more rows” WSProxy loop.

All I’m doing is pulling the “Id” field from the USA DE and matching on that (as the country code value will always be “US” for this pool of customers)

Currently, when I run the script, it times out in automation studio after 30 mins, and It updates about 5,000 rows.

I cannot re-run the script again to pick up the rest of the contacts, or automate it to run consistently, because it will always look at the same top 5,000 rows.

Right now my USA customer DE is at 80,000 but will potentially grow. This DE will only get bigger.

When I run the script in a cloud page and write the results Write("SubscriberID: " + val + "<br>"); I can see that the first 2,500 rows appear successfully but after the “Has more rows” function kicks in a counter appears between each result like so:

Code:
(Row 2,498) SubscriberID: US1111111111
(Row 2,499) SubscriberID: US1111111112
(Row 2,500) SubscriberID: US1111111113
(Row 2,501)SubscriberID: 126530 <-- ?????
(Row 2,501) SubscriberID: US1111111114
(Row 2,502)SubscriberID: 126531 <-- ?????
(Row 2,502) SubscriberID: US1111111115
(Row 2,503) SubscriberID: 126532 <-- ?????
(Row 2,503) SubscriberID: US1111111116

Which leads me to believe I’m not reading the data right, and that the WSProxy isn’t being as efficient or fast as it’s supposed to be.

Any help or pointers here would be greatly appreciated.

Here’s the full code:

Code:
<script runat="server">

Platform.Load("core", "1.1.1");

var config = {

  name: "[MY DE CUSTOMER KEY]",

  cols: ["Id"],

  filter: {

    Property: "Id",

    SimpleOperator: "isNotNull",

    Value: " "

  }

}

var records = retrieveRecords(config);

//Write(Stringify(records));

function retrieveRecords(config) {

  var prox = new Script.Util.WSProxy();

  var records = [],

    moreData = true,

    reqID = data = null;

  while (moreData) {

    moreData = false;

    if (reqID == null) {

      data = prox.retrieve("DataExtensionObject[" + config.name + "]", config.cols, config.filter);

    } else {

      data = prox.getNextBatch("DataExtensionObject[" + config.name + "]", reqID);

    }

    if (data != null) {

      moreData = data.HasMoreRows;

      reqID = data.RequestID;

      for (var i = 0; i < data.Results.length; i++) {

        var result_list = data.Results[i].Properties;

        var obj = {};

        for (k in result_list) {

          var key = result_list[k].Name;

          var val = result_list[k].Value

          if (key.indexOf("_") != 0) obj[key] = val;

//Write("ID: " + val + "<br>");

var subscriber = {

          "Attributes": {"Country_Code":"US"}};

          var subObj = Subscriber.Init(val);

          var response = subObj.Update(subscriber);

        }

        records.push(obj);

      }

    }

  }

  //return records;

}

</script>

Ten-tools.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, Ten-tools.