Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add full sharedfiles support #306

Merged
merged 23 commits into from
Jun 24, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6f97370
Add sharedfile comment support
3urobeat May 12, 2023
18011b3
Add sharedfile voting support
3urobeat May 12, 2023
d807f10
Add sharedfile subscribing support
3urobeat May 12, 2023
f09adc8
Add (disabled) sharedfile favorite support
3urobeat May 12, 2023
b55516f
Oops, wrong object name
3urobeat May 12, 2023
c92801d
Load sharedfiles component
3urobeat May 13, 2023
c864286
Add sharedfile type enum
3urobeat May 14, 2023
9ec5dcd
Add sharedfile class with fully working scraper
3urobeat May 14, 2023
d6b0fbd
Update non-object methods to take appid param
3urobeat May 14, 2023
ba27820
Misc
3urobeat May 14, 2023
350f628
Add sharedfile object methods
3urobeat May 14, 2023
c0be17c
Load CSteamSharedfile class
3urobeat May 14, 2023
8538589
Fix owner remaining null and wrong param
3urobeat May 14, 2023
81d8dc2
Use decodeSteamTime() helper instead of doing it manually
3urobeat May 14, 2023
897ad16
Formatting
3urobeat May 14, 2023
4723bd9
Improve resolveVanityURL() and move to helpers
3urobeat May 15, 2023
27d2daa
Remove steamid-resolver dep and use internal helper
3urobeat May 15, 2023
f7d7cf0
Remove support for up- & downvoting sharedfiles
3urobeat May 15, 2023
f96d25a
Rename sid to sharedFileId
3urobeat May 15, 2023
33bc8d8
Add support for determining up/downvote status
3urobeat May 28, 2023
e23fa02
Add support for reading numRatings of guides
3urobeat May 28, 2023
440b2f9
Add JsDoc
3urobeat May 29, 2023
e5fbddc
Merge branch 'master' into feature/steam-sharedfiles
3urobeat Jun 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions classes/CSteamSharedfile.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
const Cheerio = require('cheerio');
const SteamID = require('steamid');
const Helpers = require('../components/helpers.js');
const SteamCommunity = require('../index.js');
const SteamIdResolver = require('steamid-resolver');
const ESharedfileType = require('../resources/ESharedfileType.js');


/**
* Scrape a sharedfile's DOM to get all available information
* @param {String} sid - ID of the sharedfile
* @param {function} callback - First argument is null/Error, second is object containing all available information
*/
SteamCommunity.prototype.getSteamSharedfile = function(sid, callback) {

// Construct object holding all the data we can scrape
let sharedfile = {
id: sid,
type: null,
appID: null,
owner: null,
fileSize: null,
postDate: null,
resolution: null,
uniqueVisitorsCount: null,
favoritesCount: null,
upvoteCount: null
};


// Get DOM of sharedfile
this.httpRequestGet(`https://steamcommunity.com/sharedfiles/filedetails/?id=${sid}`, (err, res, body) => {
try {

/* --------------------- Preprocess output --------------------- */

// Load output into cheerio to make parsing easier
let $ = Cheerio.load(body);

// Dynamically map detailsStatsContainerLeft to detailsStatsContainerRight in an object to make readout easier. It holds size, post date and resolution.
let detailsStatsObj = {};
let detailsLeft = $(".detailsStatsContainerLeft").children();
let detailsRight = $(".detailsStatsContainerRight").children();

Object.keys(detailsLeft).forEach((e) => { // Dynamically get all details. Don't hardcore so that this also works for guides.
if (isNaN(e)) {
return; // Ignore invalid entries
}

detailsStatsObj[detailsLeft[e].children[0].data.trim()] = detailsRight[e].children[0].data;
});

// Dynamically map stats_table descriptions to values. This holds Unique Visitors and Current Favorites
let statsTableObj = {};
let statsTable = $(".stats_table").children();

Object.keys(statsTable).forEach((e) => {
if (isNaN(e)) {
return; // Ignore invalid entries
}

// Value description is at index 3, value data at index 1
statsTableObj[statsTable[e].children[3].children[0].data] = statsTable[e].children[1].children[0].data.replace(/,/g, ""); // Remove commas from 1k+ values
});


/* --------------------- Find and map values --------------------- */

// Find appID in share button onclick event
sharedfile.appID = Number($("#ShareItemBtn").attr()["onclick"].replace(`ShowSharePublishedFilePopup( '${sid}', '`, "").replace("' );", ""));


// Find fileSize if not guide
sharedfile.fileSize = detailsStatsObj["File Size"] || null; // TODO: Convert to bytes? It seems like to always be MB but no guarantee


// Find postDate and convert to timestamp
let posted = detailsStatsObj["Posted"].trim();

sharedfile.postDate = Date.parse(Helpers.decodeSteamTime(posted)); // Pass String into helper and parse the returned String to get a Unix timestamp


// Find resolution if artwork or screenshot
sharedfile.resolution = detailsStatsObj["Size"] || null;


// Find uniqueVisitorsCount. We can't use ' || null' here as Number("0") casts to false
if (statsTableObj["Unique Visitors"]) {
sharedfile.uniqueVisitorsCount = Number(statsTableObj["Unique Visitors"]);
}


// Find favoritesCount. We can't use ' || null' here as Number("0") casts to false
if (statsTableObj["Current Favorites"]) {
sharedfile.favoritesCount = Number(statsTableObj["Current Favorites"]);
}


// Find upvoteCount. We can't use ' || null' here as Number("0") casts to false
let upvoteCount = $("#VotesUpCountContainer > #VotesUpCount").text();

if (upvoteCount) {
sharedfile.upvoteCount = Number(upvoteCount);
}


// Determine type by looking at the second breadcrumb. Find the first separator as it has a unique name and go to the next element which holds our value of interest
let breadcrumb = $(".breadcrumbs > .breadcrumb_separator").next().get(0).children[0].data || "";

if (breadcrumb.includes("Screenshot")) {
sharedfile.type = ESharedfileType.Screenshot;
}

if (breadcrumb.includes("Artwork")) {
sharedfile.type = ESharedfileType.Artwork;
}

if (breadcrumb.includes("Guide")) {
sharedfile.type = ESharedfileType.Guide;
}


// Find owner profile link, convert to steamID64 using SteamIdResolver lib and create a SteamID object
let ownerHref = $(".friendBlockLinkOverlay").attr()["href"];

SteamIdResolver.customUrlToSteamID64(ownerHref, (err, steamID64) => { // This request takes <1 sec
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to avoid adding additional dependencies if not absolutely required. Vanity URL resolving already exists here; you could move this to helpers.js and use it here as well.

Copy link
Contributor Author

@3urobeat 3urobeat May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, makes sense. I have improved the function from getInventoryHistory() to both accept full URLs and only the vanity. This makes it easier to use without needing to parse the input before calling it. It should not change the functionality but I wasn't able to test it as getInventoryHistory() always returns Malformed page: no trade found for me.

I would have preferred to use the lib as it made the integration easier but I get your point. The helper should have the same functionality now.

if (!err) {
sharedfile.owner = new SteamID(steamID64);
}

// Make callback when ID was resolved as otherwise owner will always be null
callback(null, new CSteamSharedfile(this, sharedfile));
});

} catch (err) {
callback(err, null);
}
}, "steamcommunity");
};

function CSteamSharedfile(community, data) {
this._community = community;

// Clone all the data we recieved
Object.assign(this, data); // TODO: This is cleaner but might break IntelliSense. I'm leaving the block below to be reactivated if necessary

/* this.id = data.id;
this.type = data.type;
this.appID = data.appID;
this.owner = data.owner;
this.fileSize = data.fileSize;
this.postDate = data.postDate;
this.resolution = data.resolution;
this.uniqueVisitorsCount = data.uniqueVisitorsCount;
this.favoritesCount = data.favoritesCount;
this.upvoteCount = data.upvoteCount; */
}

/**
* Deletes a comment from this sharedfile's comment section
* @param {String} cid - ID of the comment to delete
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.deleteComment = function(cid, callback) {
this._community.deleteSharedfileComment(this.userID, this.id, cid, callback);
};

/**
* Favorites this sharedfile
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.favorite = function(callback) {
this._community.favoriteSharedfile(this.id, this.appID, callback);
};

/**
* Posts a comment to this sharedfile
* @param {String} message - Content of the comment to post
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.comment = function(message, callback) {
this._community.postSharedfileComment(this.owner, this.id, message, callback);
};

/**
* Subscribes to this sharedfile's comment section. Note: Checkbox on webpage does not update
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.subscribe = function(callback) {
this._community.subscribeSharedfileComments(this.owner, this.id, callback);
};

/**
* Unfavorites this sharedfile
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.unfavorite = function(callback) {
this._community.unfavoriteSharedfile(this.id, this.appID, callback);
};

/**
* Unsubscribes from this sharedfile's comment section. Note: Checkbox on webpage does not update
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.unsubscribe = function(callback) {
this._community.unsubscribeSharedfileComments(this.owner, this.id, callback);
};

/**
* Downvotes this sharedfile
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.voteDown = function(callback) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd prefer to leave voteDown and voteUp out of the library. I want to try to maintain at least a tenuous good relationship with Valve, and the potential for abuse here seems too great. I also can't really think of a valid use-case for bot voting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I removed both functions

this._community.voteDownSharedfile(this.id, callback);
};

/**
* Upvotes this sharedfile
* @param {function} callback - Takes only an Error object/null as the first argument
*/
CSteamSharedfile.prototype.voteUp = function(callback) {
this._community.voteUpSharedfile(this.id, callback);
};
Loading